Book now!

Mock Review Toolkit: Running the Simulation

On this page

4.1 Assigning Applications and At-Home Reviews

Long description

Tasks:

  • (Optional) Ability to Review Task
  • Assign Applications to Reviewers
  • Send Applications to Reviewers

Week 6: Light Simulation

Week 7-8: Full Simulation

Week 8-10: Internal Simulation

After signing the Conflict of Interest and Confidentiality agreement and receiving all Pre-Simulation Training materials, Reviewers are assigned applications for review. Reviewers can either be assigned a set of pre-selected applications or applications that fit, as best as possible, the Themes or Areas of Science they indicated during the promotion period or Reviewers may be provided with application summaries and asked to complete an Ability to Review task.

(Optional) Ability to Review Task

This optional Ability to Review task allows Reviewers to indicate the extent to which their expertise aligns with provided applications. Facilitators may decide to conduct the Ability to Review task to both better replicate CIHR’s Peer Review process and/or to try and best match Reviewer expertise to an application.

It is important to note, even if the Ability to Review process is conducted, Reviewer expertise does not need to align perfectly as this is a training exercise. Remind participants that reviewers often underappreciate their Ability to Review when reading application summaries, and that they will likely have much more to contribute than they would initially believe.

Resources

Sample Email: Application Assignment
If the Ability to Review process is conducted, this email is sent to Reviewers and lists application summaries for Reviewers to indicate their expertise alignment


Ability to Review Template
Reviewers will complete the Ability to Review task using this template, which should be attached to the Application Assignment Email.

Assign Applications to Reviewers

We recommend each Reviewer in a Light Simulation be assigned 1-2 applications, while in the Full or Internal Simulation we recommend 2-4 applications per Reviewer.

The Facilitator will also have to balance the number of Reviewers assigned to each application. In a typical Project competition, each application is assigned three Reviewers: a primary and to two secondary. The primary Reviewer is the first to give their assessment at the Committee meeting, including a brief synopsis, while secondary Reviewers concentrate on points of agreement or disagreement.

While we recommend following the standard model (1 Primary and 2 Secondary Reviewers/application), we recognize that there will many instances where the size of the committee or the number of applications available necessitates that more than 3 Reviewers be assigned to an application. We encourage the Facilitator to adapt application assignments as necessary to fit their simulation – be it having more than three Reviewers per application, having multiple primary Reviewers, or even having Reviewers with specific tasks for a given application (e.g. providing a detailed perspective on the incorporation of Sex and Gender). For additional details on the roles played by primary and secondary Reviewers during Committee, please see the Roles, Responsibilities and Scripts section later in the Toolkit.

Use the Master Planner resource to help track and assign applications. Please contact the College of Reviewers for this resource.

Send Applications to Reviewers

Mock Review Templates must be provided to Reviewers along with their assigned applications. Reviewers will conduct their reviews at-home using the provided Mock Review Templates prior to the Committee meeting. The Mock Review Template allows Reviewers to; summarize the application, provide an overall score, note strengths and weaknesses, assess sex and gender considerations, and provide a budget recommendation.

Resources

Sample Email: At-Home Reviews
Details instructions on completing at-home reviews using the Mock Review Template


Mock Review Template
Reviewers will complete the Ability to Review task using this template, which should be attached to the Application Assignment Email.


4.2 Committee Meeting

Long description

Tasks:

  • Committee Meeting Agenda and Confirmation
  • Run the Committee Meeting

Week 7: Light Simulation

Week 9-10: Full Simulation

Week 11-12: Internal Simulation

Committee Meeting Agenda and Confirmation

An email should be sent out to participants in advance of the Committee meeting to confirm the meeting’s date/ time/location and provide a meeting agenda. Facilitators can adapt the sample email and agenda to fit their specific timelines. If the meeting is virtual, we recommend Facilitators include items listed under Preparation and Room Setup below as part of their communication to participants.

Resources

Sample Email: Confirmation of Committee Meeting


Meeting Agenda Template
Must be attached to the Confirmation of Committee Meeting email. If the Committee meeting is in-person, this must also be printed for all participants – see Preparation and Room Setup below

Run the Committee Meeting

Preparation and Room Setup

Note, these are suggestions for room setup; please adapt as necessary.

If the Committee meeting is in-person, these items should be printed ahead of the meeting. If the Committee meeting is virtual, these resources can be sent to participants prior to the virtual meeting as part of the Confirmation of Committee Meeting package above.

In-meeting Resources
Number Document name Description
1 per participant Sequence of Steps for Committee Meeting A step-by-step description of the Committee meeting. Useful for all participants.
1 per participant RQA Checklist A practical tool to assist Reviewers in writing quality reviews. Useful for entire committee.
1 per Facilitator Scoring Sheet A scoring sheet that allows for tracking of scores for Facilitators OR all participants.
1 per application SO Notes Template A template for Scientific Officers to write their notes.
1 per Chair and Scientific Officer Executive Roles, Responsibilities and Scripts Scripts to assist committee executives in their role.
1 per Reviewer Reviewer Roles, Responsibilities and Scripts Scripts to assist Reviewers in their role.

If the Committee meeting is to be in-person, we suggest the table be setup as depicted below to better facilitate discussion.

Long description

Top: Reviewers

Bottom: Reviewers

Left: Scientific Officer, Chair, Facilitator

Roles, Responsibilities and Scripts

Facilitator

The Facilitator is responsible for organizing the Committee meeting(s) and ensuring that everyone has the appropriate documentation.

The Facilitator will:

Chair

It is the Chair's responsibility to ensure that the review committee functions smoothly, effectively and objectively, and that a positive, constructive, fair-minded environment in which research proposals are evaluated is established and maintained.

The Chair will:

Scientific Officer

Reviewers can rotate the role of an SO to gain experience. If this is the case, ensure that all “SO Reviewers” receive a copy of the SO Note Template (available in Preparation and Room Setup above). You can also include a Mock SO Note (included with some, but not all Mock Applications) to provide Reviewers with an example.

The Scientific Officer (SO) assumes the role of note taker. The SO will:

Reviewers

The Reviewer evaluates each of the applications assigned to them by providing a critical assessment of the applications, as well as constructive feedback based on the program's objectives and adjudication criteria described in the funding opportunity.

The Reviewer will:

Reviewer Scripts

Primary Reviewer(s) Script:

Secondary Reviewer(s) Script:

Date modified: