Critical appraisal framework for Research for Teachers

published:01 Apr 2009

When the GTC commissions a Research for Teachers (RfT) summary, the following framework is used to assess the proposal.

Pre-appraisal checks:

• Is the article in a well-known and peer refereed journal?
• Is the article relevant to one of the themes identified for current or future RfTs?
• Is the study based in at least three schools?
• Is information about the nature of the intervention given?
• Are the research methods described?
• Are the results useful – do they have the potential to be of interest or of use to people?    

    
If, on reading the article you give an unequivocal no to any question, this paper is unlikely to be an RfT summary.  Please state why below (on a scale of 1 – 4, with 1 being weak and 4 strong):

Robustness:

• Did the researchers make it clear what they wanted to know and, where appropriate, what their hypothesis was?
• Has the study built systematically on what is already known by using a literature review?
• Does the research consider evidence from more than one source, eg are data about perceptions backed up by other kinds of evidence, such as observation data, pupil assessment data, documentary analysis?
• Are conclusions and/or recommendations appropriate to the structure and size of the sample?
• Are all assertions supported by evidence?
• Have users been involved in the design or implementation of the research? Or has the research been reviewed and/or tested by users?
  Have the research questions been answered or key issues been fully examined?
• Are unanswered questions and contradictory outcomes properly dealt with?
• Was the statistical analysis valid and appropriate to the methods and aims?
• What type of study is it, based on the list of study types below?

 

Study types:

• Description
• Exploration of relationships
• Evaluation: Naturally occurring
• Evaluation: Researcher manipulated (Controlled trial?)
• Development of methodology
• Review

(Please refer to the EPPI-Centre Guidelines 0.9.7 for definitions of these study types)

Relevance:

• Is the topic relevant to the current needs of teachers, governors, parents or pupils, as these have been expressed through their representative organisations and bodies?
• Are the contexts in which the study took place broad enough to interest a range of users?
• Have the settings in the sample been adequately described?

 

Usefulness:

• Is there a clear focus on ways in which the study can impact upon policy and/or practice?
• Is the information likely to be credible and authentic to users; in particular...
• Is there detailed information about the education processes in the setting with which practitioners would be able to identify?
• Is the type and sample of research site clearly specified so that it is credible to readers?
• Is there vivid exemplification, eg through case studies with plenty of examples and illustrations to show what the intervention strategies were and how those involved responded?
• Are there implications that can be drawn from the research?

 

Writing

• Is the report lively, interesting and relatively jargon-free?
• Is the study written in such a way as to enable a good range of practitioners, including those not directly involved in the study, to relate it to their practice where this is appropriate?
• Have the findings been clearly identified and presented?
• Have the technical data and accompanying statistical analysis been presented in such a way as to be intelligible to a non-specialist reader?

 

Cautions / questions arising:

• What questions remain about the method?
• What questions remain about the findings?
• What questions will the audience ask?  (Practitioners will want to ask what they can do in their schools about the situation described in the paper.)

 

Is the title of the study sufficiently user-friendly?  (If not, please suggest an alternative title for the summary.)