Review Quality Checklist: Questions & Answers
1. What is the purpose of the Review Quality Checklist?
The Review Quality Checklist is the companion to CIHR's definition of review quality. Together, the definition and the checklist will help to harmonize review quality across CIHR's funding opportunities and set shared review quality expectations amongst applicants and reviewers.
The checklist is designed to be a practical tool to help reviewers, Chairs, Scientific Officers, and applicants apply the review quality criteria objectively. The checklist is not meant to assess the validity of a scientific opinion expressed in a review, however, as scientific opinions are addressed by other elements of the peer review process (e.g., proper application assignment, discussion amongst reviewers, oversight by Chairs).
2. How was the Review Quality Checklist developed?
The Review Quality Checklist is the product of two years of work. The definition for review quality that the checklist supports was based on the synthesis of available evidence from a scientific literature review, an international benchmarking analysis, a thorough peer review policy review, as well as consultations with the research community and an independent validation.
The criteria that form the definition of review quality have to be objective and measurable. The applicability, validity and feasibility of the indicators that form the Review Quality Checklist were developed and tested through review quality assurance pilots as well as consultations with Competition Chairs, Scientific Officers, reviewers, applicants and CIHR peer review advisory committees (e.g. College Chairs).
3. How will the Review Quality Checklist be used by CIHR?
The definition of review quality and the Review Quality Checklist will help to establish shared review quality expectations for all participants in peer review. By taking something that was rather intuitive and shaping it into a framework that allows for objective measurement, CIHR is ensuring that review quality can be assessed for any competition—regardless of size, subject matter, or peer review format.
In practice, CIHR may use the checklist indicators to monitor review quality in a given competition to inform continuous improvements to CIHR's peer review system and processes as well as to develop learning and mentoring programs to assist reviewers in conducting high-quality reviews. In addition, members of the College of Reviewers have consented to the possibility of having the quality of their reviews monitored and participating in review quality assurance process if requested by CIHR.
4. What does the Review Quality Checklist mean for reviewers?
The Review Quality Checklist is a practical tool that allows reviewers in any competition to check that their own reviews are appropriate, robust, and useful. As dedicated and knowledgeable individuals—who are often researchers themselves—we trust that reviewers will strive to provide high-quality reviews, and this checklist helps them to do so. They can use it to test any review for any competition at any time.
The definition of review quality and the Review Quality Checklist outline CIHR's expectations regarding review quality. Some (but not necessarily all) of the indicators described in the checklist will serve as a reference in the event of review quality assessments.
5. What does the Review Quality Checklist mean for applicants?
The definition of review quality and the Review Quality Checklist will help applicants understand CIHR's expectations regarding the quality of written reviews. CIHR welcomes review quality feedback from applicants to inform continuous peer review process and system improvements. The review quality checklist is not meant to assess whether the reviewer had the appropriate level of expertise to evaluate the application. Reviewer expertise is addressed as part of the peer review assignment validation process. Note that CIHR does not consider review quality concerns as suitable grounds for appeal or peer review grievance.
- Date modified: