Results of survey on the future of peer review

Community Survey Results

Total responses: 1178

1. How would you primarily classify yourself?

  • Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
  • Mid-Career Researcher (5-15 years as an independent researcher)
  • Senior Researcher (>15 years as an independent researcher)
  • Knowledge User
  • Other, please specify
  • I prefer not to answer
Answer Number Percentage
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) 151 12.82%
Mid-Career Researcher (5-15 years as an independent researcher) 375 31.83%
Senior Researcher (>15 years as an independent researcher) 537 45.59%
Knowledge User 35 2.97%
Other, please specify 45 3.82%
I prefer not to answer 35 2.97%

Total responses: 1178

Researcher please specify:

Answer Number Percentage
admin 1 3.03%
Administrateur de la recherche 1 3.03%
Administrator 1 3.03%
Bientôt en retraite de la recherche 1 3.03%
Chercheur retraité 1 3.03%
Commercialization specialist 1 3.03%
Consulting Biostatistician 1 3.03%
Director, Research Development 1 3.03%
elder knowledge keeper 1 3.03%
Emeritist Researcher 1 3.03%
étudiant 1 3.03%
Étudiant au doctorat 1 3.03%
Étudiant sénior au doctorat 1 3.03%
Étudiante au doctorat 1 3.03%
government science advisor 1 3.03%
grants advisor 1 3.03%
I am both a researcher and a knowledge user. 1 3.03%
Indigenous health scholar 1 3.03%
investigator 1 3.03%
Patent Agent 1 3.03%
Patient Partner 1 3.03%
Patient/public 1 3.03%
PhD Student 1 3.03%
Postdoc 1 3.03%
research admin 1 3.03%
Research Coordinator 1 3.03%
research director 1 3.03%
Research Policy Analyst 1 3.03%
Retired 1 3.03%
Senior Research Administrator 1 3.03%
staff support for grants 1 3.03%
Trainee 1 3.03%
Transitioning from senior researcher to retirement 1 3.03%

Total responses: 33

2. What is your preferred official language of correspondence?

  • English
  • French
  • I prefer not to answer
Answer Number Percentage
English 1010 85.59%
French 143 12.12%
I prefer not to answer 27 2.29%

Total responses: 1180

In order to inform CIHR’s decision on how peer review is conducted post-COVID, we would like to hear your thoughts on various peer review models (e.g., in person face-to-face, virtual meetings that mimic the in person face-to-face process). Please complete the following questions:

3. Have you participated in any of the following virtual review processes?

  • CIHR virtual peer review
  • Virtual peer review for another agency
  • None of the above
Total responses CIHR virtual peer review Virtual peer review for another agency None of the above
1368 Number Percentage Number Percentage Number Percentage
683 65.41% 497 47.61% 188 18.01%

4. If you have participated to a CIHR virtual peer review process, have you been satisfied with your experience?

  • Yes
    • Please indicate why
  • No
    • Please indicate what CIHR could do to improve the virtual process
  • I prefer not to answer
Total responses Yes No I prefer not to answer
497 Number Percentage Number Percentage Number Percentage
357 71.83% 128 25.75% 12 2.41%

5. If you have participated to other agency (ies) virtual peer review process, have you been satisfied with your experience?

  • Yes
    • Please indicate why
  • No
    • Please indicate why
  • I prefer not to answer
Total responses Yes No I prefer not to answer
683 Number Percentage Number Percentage Number Percentage
470 68.81% 203 29.72% 10 1.46%

6. What is your preferred model for the hosting of peer review meetings at CIHR post-COVID?

  • Virtual peer review meetings only
  • Face-to-face peer review meetings when safe to do so
  • Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
  • Other
  • I prefer not to answer
Total responses Virtual peer review meetings only Face-to-face review meetings when safe to do so Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) Other I prefer not to answer
683 Number Percent. Number Percent. Number Percent. Number Percent. Number Percent.
247 23.66% 493 47.22% 263 25.19% 35 3.35% 6 0.57%

7. Additional questions

  • Please provide any feedback (pros and cons) for the virtual peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
  • Please provide any feedback (pros and cons) for the face-to-face peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
  • Please provide any feedback (pros and cons) for the hybrid peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
Applicant Survey Results

Total responses: 627

1. How would you primarily classify yourself?

  • Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
  • Mid-Career Researcher (5-15 years as an independent researcher)
  • Senior Researcher (>15 years as an independent researcher)
  • Knowledge User
  • Other, please specify
  • I prefer not to answer
Answer Number Percentage
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) 188 28.75%
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) 213 32.57%
Mid-Career Researcher (5-15 years as an independent researcher) 214 32.72%
Senior Researcher (>15 years as an independent researcher) 1 0.15%
Knowledge User 0 0.00%
Other, please specify 11 1.68%

Total responses: 627

2. What is your preferred official language of correspondence?

  • English
  • French
  • I prefer not to answer
Answer Number Percentage
English 573 87.61%
French 46 7.03%
I prefer not to answer 8 1.22%

Total responses: 627

3. What was your level of satisfaction with the application process in the Spring 2021 Project Grant competition:

Questions

Answers

  • Very Satisfied
  • Satisfied
  • Dissatisfied
  • Not Satisfied
  • Not Applicable
  Total Resp. Very Satisfied Satisfied Unsatisfied Very Unsatisfied Not Applicable
No. Percent. No. Percent. No. Percent. No. Percent. No. Percent.
Overall Experience 631 138 21.87% 444 70.36% 35 5.55% 13 2.06% 1 0.16%
CIHR Contact Centre responsiveness to queries 637 95 15.06% 147 23.30% 21 3.33% 5 0.79% 369 58.48%
Effectiveness of CIHR communications 637 90 14.26% 396 62.76% 50 7.92% 7 1.11% 94 14.90%
Usefulness of Spring 2021 Project Grant Competition Webinars (“Ask Me Anything” and Question & Answer) 637 46 7.29% 216 34.23% 21 3.33% 8 1.27% 346 54.83%
ResearchNet system performance 637 191 30.27% 391 61.97% 37 5.86% 10 1.58% 8 1.27%
CIHR’s guidance on Integration of Sex and Gender Based Analysis 637 96 15.21% 392 62.12% 87 13.79% 33 5.23% 29 4.60%

4. CIHR is considering changes to the appendices requirements (Other Application Material) and we appreciate your feedback on the following questions:

Questions

  • Do you support the current appendices requirements (i.e.: unlimited ‘Other’ attachments with no obligation for the reviewers to read this material)?
  • Do you support the current appendices requirements (i.e.: unlimited ‘Other’attachments with no obligation for the reviewers to read this material)?
  • Do you support capping the number of attachments (e.g.: maximum of 2 document with page limits) with the requirement for reviewer read the material?

Answers

  • Yes
  • No
  • Prefer not to answer
  Total Responses Yes No Prefer not to answer
Number Percent. Number Percent. Number Percent.
Do you support the current appendices requirements (i.e.: unlimited ‘Other’ attachments with no obligation for the reviewers to read this material)? 631 347 54.99% 269 42.63% 15 2.38%
Do you support the removal of appendices (i.e.: no ‘Other’ attachments - all relevant material should be included elsewhere in the application/proposal)? 634 170 26.81% 450 70.98% 14 2.21%
Do you support capping the number of attachments (e.g.: maximum of 2 document with page limits) with the requirement for reviewer read the material? 634 320 50.47% 299 47.16% 15 2.37%

5. Based on feedback from the research community, CIHR implemented a number of changes to the Spring 2021 Project Grant competition. The following section asks about applicant use of and satisfaction with these changes.

Questions

  • Clarity of instructions on the revised Summary of Progress
  • Ease of use of the revised Summary of Progress
  • Relevance of the revised Summary of Progress

Answers

  • Very Satisfied
  • Satisfied
  • Dissatisfied
  • Not Satisfied
  • Not Applicable
  Total Responses Very Satisfied Satisfied Unsatisfied Very Unsatisfied Not Applicable
Number Percent. Number Percent. Number Percent. Number Percent. Number Percent.
Do you support the current appendices requirements (i.e.: unlimited ‘Other’ attachments with no obligation for the reviewers to read this material)? 618 90 14.56% 323 52.27% 139 22.49% 60 9.71% 6 0.97%
Do you support the removal of appendices (i.e.: no ‘Other’ attachments - all relevant material should be included elsewhere in the application/proposal)? 619 101 16.32% 344 55.57% 109 17.61% 47 7.59% 18 2.91%
Do you support capping the number of attachments (e.g.: maximum of 2 document with page limits) with the requirement for reviewer read the material? 619 133 21.49% 297 47.98% 106 17.12% 70 11.31% 13 2.10%

6. Did you use the Applicant Profile CV?

Answer Number Percentage
Yes 150 24.35%
No 423 68.67%
Prefer not to answer 43 6.98%

Total responses: 616

Question

  • Did you use the Applicant Profile CV
    • If so, please rate your level of satisfaction with the following elements:
      • Ease of use
      • Clarity of instructions

Answers

  • Very satisfied
  • Satisfied
  • Dissatisfied
  • Not Satisfied
  • Not Applicable
  Total Responses Very Satisfied Satisfied Unsatisfied Very Unsatisfied Not Applicable
No. Percent. No. Percent. No. Percent. No. Percent. No. Percent.
Ease of use 146 31 21.23% 88 60.27% 16 10.96% 10 6.85% 1 0.68%
Clarity of instructions 150 31 20.67% 93 62.00% 18 12.00% 6 4.00% 2 1.37%

7. Do you have any comments about the Applicant Profile CV?

8. Given the context of COVID-19 and the shift to virtual meetings, CIHR is would like to assess several models for the hosting of peer review meetings to inform our approach moving forward. As members of the research community, we appreciate your feedback on the following questions:

Questions: Virtual vs face-to-face experience

  • What is your preferred model for the hosting of peer review meetings?
    • Virtual peer review meetings only
    • Face-to-face peer review meetings when safe to do so
    • Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
  • Please indicate why you selected this option:
  • Do you have comments about the virtual-only model?
  • Do you have comments about the face-to-face only model?
  • Do you have comments about the hybrid model?
  • Do you have any additional feedback or suggested process improvements for CIHR?

Applicant Peer Review Responses

Total cases: 600
Total responses: 649

  Frequency Cases % Responses %
Virtual peer review meetings only 160 26.7 24.6
Face-to-face peer review meetings when safe to do so 320 53.3 49.3
Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) 169 28.2 26.0
Reviewer Survey Responses

1. How would you primarily classify yourself?

  • Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
  • Mid-Career Researcher (5-15 years as an independent researcher)
  • Senior Researcher (>15 years as an independent researcher)
  • Knowledge User
  • Other, please specify
  • I prefer not to answer
Career Stage
Answer Number Percentage
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) 35 11.63%
Mid-Career Researcher (5-15 years as an independent researcher) 104 34.55%
Senior Researcher (>15 years as an independent researcher) 154 51.16%
Knowledge User 2 0.66%
Other, please specify  5 1.66%

Total responses: 301

Researcher please specify (N=5):
Answer Number Percentage
Business development professional 1 20.00%
Clinical/administrative 1 20.00%
Patent agent (legal service provider) 1 20.00%
Technology transfer 1 20.00%
Transfer/business 1 20.00%

Total responses: 5

2. What is your preferred official language of correspondence?

  • English
  • French
  • I prefer not to answer
Answer Number Percentage
English 263 87.38%
French 35 11.63%
I prefer not to answer 3 1.00%

Total responses: 301

3. Was this your first time reviewing for CIHR’s Project Grant competition?

  • Yes
  • No
  • I prefer not to answer
Answer Number Percentage
Yes 263 87.38%
No 35 11.63%
I prefer not to answer 3 1.00%

Total responses: 297

4. Please indicate your level of satisfaction with your overall experience as a peer reviewer

Question

Please indicate your level of satisfaction with your overall experience as a peer reviewer

Answers

  • Very Satisfied
  • Satisfied
  • Dissatisfied
  • Not Satisfied
  • Not Applicable
  Total Resp. Very Satisfied Satisfied Unsatisfied Very Unsatisfied Not Applicable
Number Percent. No. Percent. No. Percent. No. Percent. No. Percent. No. Percent.
Please indicate your level of satisfaction with your overall experience as a peer reviewer: 297 85 28.62% 177 59.60% 20 6.73% 11 3.70% 4 1.35%

5. Did you attend the committee pre-meeting session?

  • Yes
    • If yes, is there anything that you would improve, add or change?
  • No
    • If not, please indicate the reason you could not attend, or if there was anything CIHR could have done to better accommodate your attendance?
  • I prefer not to answer
Answer Number Percentage
Yes 280 94.28%
No 58 19.53%
I prefer not to answer 2 0.67%

Total responses: 297

6. Were you satisfied with the clarity of reviewer instruction provided in the reviewer manual?

  • Yes
  • No
    • If you were not satisfied, is there a specific section or set of instructions which you would suggest improving?
  • I prefer not to answer
Answer Number Percentage
Yes 237 79.80%
No 14 3.70%
I prefer not to answer 3 1.01%

Total responses: 297

7. Were you satisfied with the overall preparation/training in the session provided through your interaction with CIHR staff, the reviewer manual, reviewer webinar and committee pre-meeting?

  • Yes
  • No
    • If not, please indicate what could be done to improve
  • I prefer not to answer
Answer Number Percentage
Yes 281 94.61%
No 11 4.71%
I prefer not to answer 5 1.68%

Total responses: 297

8. Were you satisfied with the responsiveness to queries/concerns?

  • Yes
  • No
    • If not, please indicate what could be done to improve
  • I prefer not to answer
Answer Number Percentage
Yes 287 96.63%
No 4 1.35%
I prefer not to answer 6 2.02%

Total responses: 297

9. Do you support the complete withdrawal of applications that do not follow the basic guidelines associated with the funding opportunity? This includes formatting (font size, spacing, margins, number of pages)?

  • Yes
    • Please provide rationale for your answer
  • No
    • Please provide rationale for your answer
  • I prefer not to answer
Answer Number Percentage
Yes 253 86.64%
No 32 10.96%
I prefer not to answer 7 2.40%

Total responses: 292

10. As you know, a new Summary of Progress was added to the application process. Was this helpful  in your review?

  • Yes
    • If yes, please explain why the summary of progress was useful in your review
  • No
    • If no, please explain why the summary of progress was not useful in your review
  • I prefer not to answer
Answer Number Percentage
Yes 255 86.15%
No 89 30.48%
I prefer not to answer 14 4.79%

Total responses: 292

11. What changes to the current use of attachments of other application material in the appendices would you support?

  • Current format is fine – leave it to reviewers to access if necessary
  • No appendices as the 10 page proposal should contain all relevant data
  • Reduce the appendices from the current 5 to a lesser number
  • I prefer not to answer
  Total Responses Current format is fine - leave it to reviewers to access if necessary No appendices as the 10 page should contain all relevant data Reduce the appendices from the current 5 to a lesser number I prefer not to answer
No. Percent. No. Percent. No. Percent. No. Percent. No. Percent.
What changes to the current use of attachment of other application material in the appendices would you support? 292 162 55.48% 78 26.71% 47 16.10% 5 1.71%

12. Would you agree to be a mentor for the Reviewer in Training program?

  • Yes
    • If yes, please indicate why
  • No
    • If no, please indicate if there is anything CIHR could do to improve to allow you to participate
  • I prefer not to answer
Answer Number Percentage
Yes 152 52.05%
No 106 36.30%
I prefer not to answer 5 1.71%

Total responses: 292

13. Were you satisfied with the compensation for dependent care and other costs (if applicable) accrued as a result of participation in CIHR peer review?

  • Yes
  • No
    • If you were not satisfied, please indicate why
  • Not applicable
  • I prefer not to answer
Answer Number Percentage
Yes 38 12.84%
No 13 4.39%
I prefer not to answer 5 1.69%
Not applicable 240 81.08%

Total responses: 296

14. Were you satisfied with how meetings were Chaired?

  • Yes
  • No
    • If you were not satisfied, please indicate why
  • I prefer not to answer
Answer Number Percentage
Yes 255 86.15%
No 24 8.11%
I prefer not to answer 17 5.74%

Total responses: 296

15. Were you satisfied with the quality of Scientific Officer summaries offered at the meeting?

  • Yes
  • No
    • If you were not satisfied, please indicate why
  • I prefer not to answer
Answer Number Percentage
Yes 273 92.29%
No 11 3.72%
I prefer not to answer 12 4.05%

Total responses: 296

16. In order to inform CIHR’s decision on how peer review is conducted post-COVID, we would like to hear your thoughts on various peer review models (e.g., in person face-to-face, virtual meetings that mimic the in person face-to-face process). Please complete the following questions;

  • Have you been satisfied with your experience with CIHR virtual peer review processes to date?
  • Yes
    • Please indicate why
  • No
    • Please indicate what CIHR could do to improve the virtual process
  • I prefer not to answer
Answer Number Percentage
Yes 230 79.86%
No 52 18.06%
I prefer not to answer 6 2.08%

Total responses: 288

17. Have you participated to other agency (ies) virtual peer review process?

  • Yes
  • No
  • I prefer not to answer
Answer Number Percentage
Yes 165 57.29%
No 120 41.67%
I prefer not to answer 3 1.04%

Total responses: 288

  • If yes, have you been satisfied with your experience?
    • Yes
      • Please indicate why
    • No
      • Please indicate why
Answer Number Percentage
Yes 132 80.00%
No 29 17.58%
I prefer not to answer 4 2.42%

Total responses: 165

18. What is your preferred model for the hosting of peer review meetings at CIHR post-COVID?

  • Virtual peer review meetings only
  • Face-to-face peer review meetings when safe to do so
  • Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
  • Other (Please indicate why you selected this option.)
  Virtual peer review meetings only Face-to-face peer review meetings when safe to do so Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) Other I prefer not to answer
No. Percent. No. Percent. No. Percent. No. Percent. No. Percent.
What is your preferred model for the hosting of peer review meetings at CIHR post-COVID? 63 21.88% 148 51.39% 67 23.26% 5 1.71% 5 1.74%

Total responses: 288

19. Remaining survey questions

  • Please provide any feedback (pros and cons) for the virtual peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
  • Please provide any feedback (pros and cons) for the face-to-face peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
  • Please provide any feedback (pros and cons) for the hybrid peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
  • Do you have any additional feedback or suggested process improvements for CIHR?
Date modified: