Results of survey on the future of peer review
Community Survey Results
Total responses: 1178
1. How would you primarily classify yourself?
- Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
- Mid-Career Researcher (5-15 years as an independent researcher)
- Senior Researcher (>15 years as an independent researcher)
- Knowledge User
- Other, please specify
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) | 151 | 12.82% |
Mid-Career Researcher (5-15 years as an independent researcher) | 375 | 31.83% |
Senior Researcher (>15 years as an independent researcher) | 537 | 45.59% |
Knowledge User | 35 | 2.97% |
Other, please specify | 45 | 3.82% |
I prefer not to answer | 35 | 2.97% |
Total responses: 1178
Researcher please specify:
Answer | Number | Percentage |
---|---|---|
admin | 1 | 3.03% |
Administrateur de la recherche | 1 | 3.03% |
Administrator | 1 | 3.03% |
Bientôt en retraite de la recherche | 1 | 3.03% |
Chercheur retraité | 1 | 3.03% |
Commercialization specialist | 1 | 3.03% |
Consulting Biostatistician | 1 | 3.03% |
Director, Research Development | 1 | 3.03% |
elder knowledge keeper | 1 | 3.03% |
Emeritist Researcher | 1 | 3.03% |
étudiant | 1 | 3.03% |
Étudiant au doctorat | 1 | 3.03% |
Étudiant sénior au doctorat | 1 | 3.03% |
Étudiante au doctorat | 1 | 3.03% |
government science advisor | 1 | 3.03% |
grants advisor | 1 | 3.03% |
I am both a researcher and a knowledge user. | 1 | 3.03% |
Indigenous health scholar | 1 | 3.03% |
investigator | 1 | 3.03% |
Patent Agent | 1 | 3.03% |
Patient Partner | 1 | 3.03% |
Patient/public | 1 | 3.03% |
PhD Student | 1 | 3.03% |
Postdoc | 1 | 3.03% |
research admin | 1 | 3.03% |
Research Coordinator | 1 | 3.03% |
research director | 1 | 3.03% |
Research Policy Analyst | 1 | 3.03% |
Retired | 1 | 3.03% |
Senior Research Administrator | 1 | 3.03% |
staff support for grants | 1 | 3.03% |
Trainee | 1 | 3.03% |
Transitioning from senior researcher to retirement | 1 | 3.03% |
Total responses: 33
2. What is your preferred official language of correspondence?
- English
- French
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
English | 1010 | 85.59% |
French | 143 | 12.12% |
I prefer not to answer | 27 | 2.29% |
Total responses: 1180
In order to inform CIHR’s decision on how peer review is conducted post-COVID, we would like to hear your thoughts on various peer review models (e.g., in person face-to-face, virtual meetings that mimic the in person face-to-face process). Please complete the following questions:
3. Have you participated in any of the following virtual review processes?
- CIHR virtual peer review
- Virtual peer review for another agency
- None of the above
Total responses | CIHR virtual peer review | Virtual peer review for another agency | None of the above | |||
---|---|---|---|---|---|---|
1368 | Number | Percentage | Number | Percentage | Number | Percentage |
683 | 65.41% | 497 | 47.61% | 188 | 18.01% |
4. If you have participated to a CIHR virtual peer review process, have you been satisfied with your experience?
- Yes
- Please indicate why
- No
- Please indicate what CIHR could do to improve the virtual process
- I prefer not to answer
Total responses | Yes | No | I prefer not to answer | |||
---|---|---|---|---|---|---|
497 | Number | Percentage | Number | Percentage | Number | Percentage |
357 | 71.83% | 128 | 25.75% | 12 | 2.41% |
5. If you have participated to other agency (ies) virtual peer review process, have you been satisfied with your experience?
- Yes
- Please indicate why
- No
- Please indicate why
- I prefer not to answer
Total responses | Yes | No | I prefer not to answer | |||
---|---|---|---|---|---|---|
683 | Number | Percentage | Number | Percentage | Number | Percentage |
470 | 68.81% | 203 | 29.72% | 10 | 1.46% |
6. What is your preferred model for the hosting of peer review meetings at CIHR post-COVID?
- Virtual peer review meetings only
- Face-to-face peer review meetings when safe to do so
- Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
- Other
- I prefer not to answer
Total responses | Virtual peer review meetings only | Face-to-face review meetings when safe to do so | Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) | Other | I prefer not to answer | |||||
---|---|---|---|---|---|---|---|---|---|---|
683 | Number | Percent. | Number | Percent. | Number | Percent. | Number | Percent. | Number | Percent. |
247 | 23.66% | 493 | 47.22% | 263 | 25.19% | 35 | 3.35% | 6 | 0.57% |
7. Additional questions
- Please provide any feedback (pros and cons) for the virtual peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
- Please provide any feedback (pros and cons) for the face-to-face peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
- Please provide any feedback (pros and cons) for the hybrid peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
Applicant Survey Results
Total responses: 627
1. How would you primarily classify yourself?
- Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
- Mid-Career Researcher (5-15 years as an independent researcher)
- Senior Researcher (>15 years as an independent researcher)
- Knowledge User
- Other, please specify
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) | 188 | 28.75% |
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) | 213 | 32.57% |
Mid-Career Researcher (5-15 years as an independent researcher) | 214 | 32.72% |
Senior Researcher (>15 years as an independent researcher) | 1 | 0.15% |
Knowledge User | 0 | 0.00% |
Other, please specify | 11 | 1.68% |
Total responses: 627
2. What is your preferred official language of correspondence?
- English
- French
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
English | 573 | 87.61% |
French | 46 | 7.03% |
I prefer not to answer | 8 | 1.22% |
Total responses: 627
3. What was your level of satisfaction with the application process in the Spring 2021 Project Grant competition:
Questions
- Overall Experience
- CIHR Contact Centre responsiveness to queries
- Effectiveness of CIHR communications
- Usefulness of Spring 2021 Project Grant Competition Webinars (“Ask Me Anything” and Question & Answer)
- ResearchNet system performance
- CIHR’s guidance on Integration of Sex and Gender Based Analysis
Answers
- Very Satisfied
- Satisfied
- Dissatisfied
- Not Satisfied
- Not Applicable
Total Resp. | Very Satisfied | Satisfied | Unsatisfied | Very Unsatisfied | Not Applicable | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | ||
Overall Experience | 631 | 138 | 21.87% | 444 | 70.36% | 35 | 5.55% | 13 | 2.06% | 1 | 0.16% |
CIHR Contact Centre responsiveness to queries | 637 | 95 | 15.06% | 147 | 23.30% | 21 | 3.33% | 5 | 0.79% | 369 | 58.48% |
Effectiveness of CIHR communications | 637 | 90 | 14.26% | 396 | 62.76% | 50 | 7.92% | 7 | 1.11% | 94 | 14.90% |
Usefulness of Spring 2021 Project Grant Competition Webinars (“Ask Me Anything” and Question & Answer) | 637 | 46 | 7.29% | 216 | 34.23% | 21 | 3.33% | 8 | 1.27% | 346 | 54.83% |
ResearchNet system performance | 637 | 191 | 30.27% | 391 | 61.97% | 37 | 5.86% | 10 | 1.58% | 8 | 1.27% |
CIHR’s guidance on Integration of Sex and Gender Based Analysis | 637 | 96 | 15.21% | 392 | 62.12% | 87 | 13.79% | 33 | 5.23% | 29 | 4.60% |
4. CIHR is considering changes to the appendices requirements (Other Application Material) and we appreciate your feedback on the following questions:
Questions
- Do you support the current appendices requirements (i.e.: unlimited ‘Other’ attachments with no obligation for the reviewers to read this material)?
- Do you support the current appendices requirements (i.e.: unlimited ‘Other’attachments with no obligation for the reviewers to read this material)?
- Do you support capping the number of attachments (e.g.: maximum of 2 document with page limits) with the requirement for reviewer read the material?
Answers
- Yes
- No
- Prefer not to answer
Total Responses | Yes | No | Prefer not to answer | ||||
---|---|---|---|---|---|---|---|
Number | Percent. | Number | Percent. | Number | Percent. | ||
Do you support the current appendices requirements (i.e.: unlimited ‘Other’ attachments with no obligation for the reviewers to read this material)? | 631 | 347 | 54.99% | 269 | 42.63% | 15 | 2.38% |
Do you support the removal of appendices (i.e.: no ‘Other’ attachments - all relevant material should be included elsewhere in the application/proposal)? | 634 | 170 | 26.81% | 450 | 70.98% | 14 | 2.21% |
Do you support capping the number of attachments (e.g.: maximum of 2 document with page limits) with the requirement for reviewer read the material? | 634 | 320 | 50.47% | 299 | 47.16% | 15 | 2.37% |
5. Based on feedback from the research community, CIHR implemented a number of changes to the Spring 2021 Project Grant competition. The following section asks about applicant use of and satisfaction with these changes.
Questions
- Clarity of instructions on the revised Summary of Progress
- Ease of use of the revised Summary of Progress
- Relevance of the revised Summary of Progress
Answers
- Very Satisfied
- Satisfied
- Dissatisfied
- Not Satisfied
- Not Applicable
Total Responses | Very Satisfied | Satisfied | Unsatisfied | Very Unsatisfied | Not Applicable | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Number | Percent. | Number | Percent. | Number | Percent. | Number | Percent. | Number | Percent. | ||
Do you support the current appendices requirements (i.e.: unlimited ‘Other’ attachments with no obligation for the reviewers to read this material)? | 618 | 90 | 14.56% | 323 | 52.27% | 139 | 22.49% | 60 | 9.71% | 6 | 0.97% |
Do you support the removal of appendices (i.e.: no ‘Other’ attachments - all relevant material should be included elsewhere in the application/proposal)? | 619 | 101 | 16.32% | 344 | 55.57% | 109 | 17.61% | 47 | 7.59% | 18 | 2.91% |
Do you support capping the number of attachments (e.g.: maximum of 2 document with page limits) with the requirement for reviewer read the material? | 619 | 133 | 21.49% | 297 | 47.98% | 106 | 17.12% | 70 | 11.31% | 13 | 2.10% |
6. Did you use the Applicant Profile CV?
Answer | Number | Percentage |
---|---|---|
Yes | 150 | 24.35% |
No | 423 | 68.67% |
Prefer not to answer | 43 | 6.98% |
Total responses: 616
Question
- Did you use the Applicant Profile CV
- If so, please rate your level of satisfaction with the following elements:
- Ease of use
- Clarity of instructions
- If so, please rate your level of satisfaction with the following elements:
Answers
- Very satisfied
- Satisfied
- Dissatisfied
- Not Satisfied
- Not Applicable
Total Responses | Very Satisfied | Satisfied | Unsatisfied | Very Unsatisfied | Not Applicable | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | ||
Ease of use | 146 | 31 | 21.23% | 88 | 60.27% | 16 | 10.96% | 10 | 6.85% | 1 | 0.68% |
Clarity of instructions | 150 | 31 | 20.67% | 93 | 62.00% | 18 | 12.00% | 6 | 4.00% | 2 | 1.37% |
7. Do you have any comments about the Applicant Profile CV?
8. Given the context of COVID-19 and the shift to virtual meetings, CIHR is would like to assess several models for the hosting of peer review meetings to inform our approach moving forward. As members of the research community, we appreciate your feedback on the following questions:
Questions: Virtual vs face-to-face experience
-
What is your preferred model for the hosting of peer review meetings?
- Virtual peer review meetings only
- Face-to-face peer review meetings when safe to do so
- Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
- Please indicate why you selected this option:
- Do you have comments about the virtual-only model?
- Do you have comments about the face-to-face only model?
- Do you have comments about the hybrid model?
- Do you have any additional feedback or suggested process improvements for CIHR?
Applicant Peer Review Responses
Total cases: 600
Total responses: 649
Frequency | Cases % | Responses % | |
---|---|---|---|
Virtual peer review meetings only | 160 | 26.7 | 24.6 |
Face-to-face peer review meetings when safe to do so | 320 | 53.3 | 49.3 |
Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) | 169 | 28.2 | 26.0 |
Reviewer Survey Responses
1. How would you primarily classify yourself?
- Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
- Mid-Career Researcher (5-15 years as an independent researcher)
- Senior Researcher (>15 years as an independent researcher)
- Knowledge User
- Other, please specify
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Early Career Researcher (<5 years as an independent researcher; note: ECR status extended by 1 year) | 35 | 11.63% |
Mid-Career Researcher (5-15 years as an independent researcher) | 104 | 34.55% |
Senior Researcher (>15 years as an independent researcher) | 154 | 51.16% |
Knowledge User | 2 | 0.66% |
Other, please specify | 5 | 1.66% |
Total responses: 301
Answer | Number | Percentage |
---|---|---|
Business development professional | 1 | 20.00% |
Clinical/administrative | 1 | 20.00% |
Patent agent (legal service provider) | 1 | 20.00% |
Technology transfer | 1 | 20.00% |
Transfer/business | 1 | 20.00% |
Total responses: 5
2. What is your preferred official language of correspondence?
- English
- French
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
English | 263 | 87.38% |
French | 35 | 11.63% |
I prefer not to answer | 3 | 1.00% |
Total responses: 301
3. Was this your first time reviewing for CIHR’s Project Grant competition?
- Yes
- No
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 263 | 87.38% |
No | 35 | 11.63% |
I prefer not to answer | 3 | 1.00% |
Total responses: 297
4. Please indicate your level of satisfaction with your overall experience as a peer reviewer
Question
Please indicate your level of satisfaction with your overall experience as a peer reviewer
Answers
- Very Satisfied
- Satisfied
- Dissatisfied
- Not Satisfied
- Not Applicable
Total Resp. | Very Satisfied | Satisfied | Unsatisfied | Very Unsatisfied | Not Applicable | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Number | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. |
Please indicate your level of satisfaction with your overall experience as a peer reviewer: | 297 | 85 | 28.62% | 177 | 59.60% | 20 | 6.73% | 11 | 3.70% | 4 | 1.35% |
5. Did you attend the committee pre-meeting session?
- Yes
- If yes, is there anything that you would improve, add or change?
- No
- If not, please indicate the reason you could not attend, or if there was anything CIHR could have done to better accommodate your attendance?
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 280 | 94.28% |
No | 58 | 19.53% |
I prefer not to answer | 2 | 0.67% |
Total responses: 297
6. Were you satisfied with the clarity of reviewer instruction provided in the reviewer manual?
- Yes
- No
- If you were not satisfied, is there a specific section or set of instructions which you would suggest improving?
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 237 | 79.80% |
No | 14 | 3.70% |
I prefer not to answer | 3 | 1.01% |
Total responses: 297
7. Were you satisfied with the overall preparation/training in the session provided through your interaction with CIHR staff, the reviewer manual, reviewer webinar and committee pre-meeting?
- Yes
- No
- If not, please indicate what could be done to improve
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 281 | 94.61% |
No | 11 | 4.71% |
I prefer not to answer | 5 | 1.68% |
Total responses: 297
8. Were you satisfied with the responsiveness to queries/concerns?
- Yes
- No
- If not, please indicate what could be done to improve
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 287 | 96.63% |
No | 4 | 1.35% |
I prefer not to answer | 6 | 2.02% |
Total responses: 297
9. Do you support the complete withdrawal of applications that do not follow the basic guidelines associated with the funding opportunity? This includes formatting (font size, spacing, margins, number of pages)?
- Yes
- Please provide rationale for your answer
- No
- Please provide rationale for your answer
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 253 | 86.64% |
No | 32 | 10.96% |
I prefer not to answer | 7 | 2.40% |
Total responses: 292
10. As you know, a new Summary of Progress was added to the application process. Was this helpful in your review?
- Yes
- If yes, please explain why the summary of progress was useful in your review
- No
- If no, please explain why the summary of progress was not useful in your review
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 255 | 86.15% |
No | 89 | 30.48% |
I prefer not to answer | 14 | 4.79% |
Total responses: 292
11. What changes to the current use of attachments of other application material in the appendices would you support?
- Current format is fine – leave it to reviewers to access if necessary
- No appendices as the 10 page proposal should contain all relevant data
- Reduce the appendices from the current 5 to a lesser number
- I prefer not to answer
Total Responses | Current format is fine - leave it to reviewers to access if necessary | No appendices as the 10 page should contain all relevant data | Reduce the appendices from the current 5 to a lesser number | I prefer not to answer | |||||
---|---|---|---|---|---|---|---|---|---|
No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. |
What changes to the current use of attachment of other application material in the appendices would you support? | 292 | 162 | 55.48% | 78 | 26.71% | 47 | 16.10% | 5 | 1.71% |
12. Would you agree to be a mentor for the Reviewer in Training program?
- Yes
- If yes, please indicate why
- No
- If no, please indicate if there is anything CIHR could do to improve to allow you to participate
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 152 | 52.05% |
No | 106 | 36.30% |
I prefer not to answer | 5 | 1.71% |
Total responses: 292
13. Were you satisfied with the compensation for dependent care and other costs (if applicable) accrued as a result of participation in CIHR peer review?
- Yes
- No
- If you were not satisfied, please indicate why
- Not applicable
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 38 | 12.84% |
No | 13 | 4.39% |
I prefer not to answer | 5 | 1.69% |
Not applicable | 240 | 81.08% |
Total responses: 296
14. Were you satisfied with how meetings were Chaired?
- Yes
- No
- If you were not satisfied, please indicate why
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 255 | 86.15% |
No | 24 | 8.11% |
I prefer not to answer | 17 | 5.74% |
Total responses: 296
15. Were you satisfied with the quality of Scientific Officer summaries offered at the meeting?
- Yes
- No
- If you were not satisfied, please indicate why
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 273 | 92.29% |
No | 11 | 3.72% |
I prefer not to answer | 12 | 4.05% |
Total responses: 296
16. In order to inform CIHR’s decision on how peer review is conducted post-COVID, we would like to hear your thoughts on various peer review models (e.g., in person face-to-face, virtual meetings that mimic the in person face-to-face process). Please complete the following questions;
- Have you been satisfied with your experience with CIHR virtual peer review processes to date?
- Yes
- Please indicate why
- No
- Please indicate what CIHR could do to improve the virtual process
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 230 | 79.86% |
No | 52 | 18.06% |
I prefer not to answer | 6 | 2.08% |
Total responses: 288
17. Have you participated to other agency (ies) virtual peer review process?
- Yes
- No
- I prefer not to answer
Answer | Number | Percentage |
---|---|---|
Yes | 165 | 57.29% |
No | 120 | 41.67% |
I prefer not to answer | 3 | 1.04% |
Total responses: 288
- If yes, have you been satisfied with your experience?
- Yes
- Please indicate why
- No
- Please indicate why
- Yes
Answer | Number | Percentage |
---|---|---|
Yes | 132 | 80.00% |
No | 29 | 17.58% |
I prefer not to answer | 4 | 2.42% |
Total responses: 165
18. What is your preferred model for the hosting of peer review meetings at CIHR post-COVID?
- Virtual peer review meetings only
- Face-to-face peer review meetings when safe to do so
- Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
- Other (Please indicate why you selected this option.)
Virtual peer review meetings only | Face-to-face peer review meetings when safe to do so | Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) | Other | I prefer not to answer | ||||||
---|---|---|---|---|---|---|---|---|---|---|
No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | No. | Percent. | |
What is your preferred model for the hosting of peer review meetings at CIHR post-COVID? | 63 | 21.88% | 148 | 51.39% | 67 | 23.26% | 5 | 1.71% | 5 | 1.74% |
Total responses: 288
19. Remaining survey questions
- Please provide any feedback (pros and cons) for the virtual peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
- Please provide any feedback (pros and cons) for the face-to-face peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
- Please provide any feedback (pros and cons) for the hybrid peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
- Do you have any additional feedback or suggested process improvements for CIHR?
- Date modified: