Criteria for validating assessment tools
The GRADE approach is used to assess the quality of evidence for a specific outcome across studies.
The grading of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in health technology assessment and guideline development organisations throughout the world.
GRADE provides a transparent approach to reaching judgements about the quality of evidence on the effects of a health care intervention, but is complex and therefore challenging to apply in a consistent manner.
We developed a checklist to guide the researcher to extract the data required to make a GRADE assessment.
We applied the checklist to 29 meta-analyses of randomised controlled trials on the effectiveness of health care interventions.
Two reviewers used the checklist for each paper and used these data to rate the quality of evidence for a particular outcome.
There was consistent agreement between reviewers on most items in the checklist.
The use of this checklist may be an aid to improving the consistency and reproducibility of GRADE assessments, particularly for inexperienced users or in rapid reviews without the resources to conduct assessments by two researchers independently. This approach has been widely implemented by organisations such as the World Health Organization, Cochrane Collaboration, Agency for Healthcare Research and Quality (USA) and National Institute of Health and Care Excellence (UK).
The GRADE approach is comprehensively described in an online manual (freely available for download with the GRADEpro software at ].
Upgrading occurs for three primary reasons: large magnitude of effect, evidence of a dose-response effect and all plausible confounding taken into account.
After the process of downgrading or upgrading, the quality of the evidence for each outcome ends up rated as high, moderate, low or very low.
For the purposes of this paper, we will focus on the application of GRADE to meta-analyses of RCTs.