Virtual Peer Review Guide – Strategy for Patient-Oriented Research (SPOR) Networks in Chronic Disease

Table of Contents

1. Introduction

The aim of this document is to serve as a resource for the adjudication of the SPOR Networks in Chronic Disease applications.

2. Highlights of the Funding Opportunity

2.1 Background

Patient-oriented research, a cornerstone of evidence-informed health care, refers to a continuum of research, from initial studies in humans to comparative effectiveness and outcomes research, and the integration of this research into the health care system and clinical practice.

Canada’s Strategy for Patient-Oriented Research (SPOR) is a strategy that puts patients first. The vision of the strategy is that Canada will demonstrably improve health outcomes and enhance the health care experience for patients through the integration of evidence at all levels of the health care system.

This funding opportunity will provide support for the development and implementation of SPOR Networks in non-communicable chronic diseases. For the purpose of this opportunity, non-communicable chronic disease is defined as a long-term disease that is not passed from one person to another; it evolves over time, often progressing in severity. Although a non-communicable chronic disease may be prevented and can often be controlled, it is rarely cured.

2.2 Objectives

The primary objective of the opportunity will be to translate existing and new knowledge generated by basic biomedical, clinical, and population health research into testing of innovations that can improve clinical science and practice and foster policy changes, leading to transformative and measureable improvements in patient health outcomes, and in greater efficiency and effectiveness in healthcare delivery within five years.

Network applicants are expected to show that the planned program of research and knowledge translation will have a significant impact on non-communicable chronic diseases and their co-morbidities by:

  • Addressing patient-oriented research priorities and developing an interdisciplinary nation-wide agenda of outcomes-oriented research.
  • Accelerating the translation of existing and new knowledge generated by basic biomedical, clinical and population health research into clinical science and practice and policy applications.
  • Strengthening the capacity to conduct patient-oriented research through an integrated multidisciplinary program that provides unique and innovative training and mentoring opportunities.
  • Fostering the development and implementation of evidence-informed and gender-responsive practices, policies, services, products and programs to improve patient outcomes, access to care and quality, efficiency and effectiveness of health care.

2.3 Focus of the Networks

While Canada has a strong foundation on which to build, a number of factors impede the flow of information from basic biomedical research through clinical research to changes in patient care and improved health outcomes.

To address this issue, applicants are expected to:

  • Focus on translating knowledge generated by basic biomedical, clinical and population health research into the clinical research phase where it will have significant, real-world impact. Note that moving what is gained from clinical research into policy and practice is within scope but should not be the principle focus of the proposal.
  • Address first-in-human and beyond research in non-communicable chronic diseases that have resulted in areas of high disease burden.
  • Include meaningful representation from each of the following stakeholder groups: 1) researchers; 2) health care professionals; 3) patients/family representatives/informal care providers; and 4) health policy makers.
  • Demonstrate that it will be pan-Canadian in scope, where key applicant(s) are from three of the following four regions:
    • East – includes Newfoundland and Labrador, Nova Scotia, Prince Edward Island and New Brunswick
    • Central – includes Quebec, Ontario and Manitoba
    • West – includes Saskatchewan, Alberta and British Columbia
    • North – includes Nunavut, Northwest Territories and Yukon

In addition, applicants are strongly encouraged to:

  • Explore the opportunities for impact across multiple non-communicable chronic diseases.
  • Include relevant international collaborations.

Note: Primary prevention is not considered to be in scope for this funding opportunity.

2.4 Funds Available

  • $49.8M is available to fund up to four distinct networks.
  • Each successful network will receive up to $12,450,000 over five years from CIHR and must be matched at a minimum of 1:1 with funding from non-federal sources.

3. Review Process

There will be a two stage review process to adjudicate the applications: a remote (virtual) review of all applications followed by a face-to-face meeting of a Selection Panel. The stage you are reviewing for is the virtual stage.

Six reviewers will be assigned to each application covering all key stakeholder groups, including patients, patient engagement experts, clinicians, researchers and policy experts. For each application assigned, reviewers are asked to provide an assessment of the strengths and weaknesses for each of the evaluation criteria outlined in section 3.1 as well as to complete the “Summary of Assessment” in the evaluation form provided. It is important for reviewers to clearly articulate the strengths and weaknesses taking all of the criteria into consideration as they will be used for three purposes: 1) to provide applicants with feedback, 2) to provide the other reviewers assigned to the application with details of the assessment, and 3) to provide the Selection Panel in the next stage with a comprehensive review. It is important to note that the Selection Panel wil not be reviewing the applications in depth. They will be using the reviews from the virtual stage to rank and ultimately select four applications in distinct areas for funding.

All six reviewers assigned to an application will participate in a brief teleconference to discuss and understand discrepancies in reviews and budget recommendations (if applicable). There will be no requirement to reach consensus. Once the teleconference is complete, reviewers will then finalize their reviews as required. At this point, the virtual adjudication process will be complete.

All reviews will have to go to a Selection Panel by January 8, 2016 to give the Selection Panel sufficient time to prepare for their meeting.

Final Reviews Submitted – December 18, 2015
Teleconferences – from December 18, 2015 to January 7, 2016

3.1 Evaluation Criteria

  1. Vision, rationale, priorities and added value
    1. Coherence of the Network’s vision and rationale, with a focus on accelerating the translation of knowledge into clinical research and practice.
    2. Appropriateness of the patient*-driven priority-setting mechanism(s)/process(es) that will inform the program of research.
    3. Clarity of the expected outcomes and the added value the Network will bring in realizing these outcomes.
    4. Likelihood of impact on clinical practice within the timeframe of the Network funding.
    5. Clarity of the Network’s vision regarding analysis and expected outcomes as they relate to sex and gender, ethnicity, and other key social determinants of health, including geographic and economic.
    6. Evidence of explicit consideration being given to achieving equitable health impacts across patient/population sub-groups.
    7. Where applicable, the value-add of the cross-cutting nature of the proposal (i.e., the value-add of addressing multiple chronic diseases).
  2. Feasibility, transformative impact and sustainability
    1. Creativity and innovativeness of the Network strategy, and the transformational potential for the clinical care of patients.
    2. Degree to which the proposal addresses chronic disease(s) that have resulted in areas of high disease burden and/or high healthcare costs to Canadian society.
    3. Demonstration that the Network is pan-Canadian in scope and design in such a way that it has the potential to significantly improve the lives of patients in three of the four designated regions in Canada: East, Central, West, North.  Differential needs of patient populations in these geographic areas, and differential clinical service models are explicitly taken into account in the design of the Network.
    4. Quality and relevance of the stakeholder engagement strategy including attention to engagement of vulnerable populations.
    5. Quality and potential effectiveness of the Network’s sustainability plan.
    6. Quality of the partnerships and their potential to achieve outcomes and long-term sustainability.
    7. Evidence of leveraging existing resources and platforms, including SPOR SUPPORT Units.
    8. Relevance of self-assessed barriers and enablers to the Network strategy and appropriateness of the plan to address them, including a clear articulation of key risks and suitable mitigation strategies.
    9. Relevance of national synergies and, if applicable, international linkages.
    10. Quality of the performance measurement framework, including relevant metrics, to evaluate outcomes that matter most to patients, their healthcare providers, and health policy makers.
  3. Patient-oriented research program
    1. Quality of the proposed research program and the extent to which it fits in the Network’s overall vision to address challenges and opportunities.
    2. Appropriateness of the research approach, i.e., the underlying hypotheses and/or propositions, research questions, design, research methods, as well as a realistic applicant assessment of limitations and alternatives.
    3. Evidence of appropriate and meaningful integration of patient engagement mechanisms to ensure that the program of research has been informed by patients from the onset and will continue to be informed by patients.
    4. Completeness of the literature review and its relevance to supporting the theoretical and/or conceptual framework for the proposed research program.
    5. The relevance of the research program for end users (e.g., patients, health service providers, health policy makers, etc.) and the extent to which the research program reflects patient priorities and is focused on improving patient outcomes.
    6. Quality and appropriateness of the strategy for addressing issues related to sex and gender, ethnicity, and social determinants of health, including geographic and economic. 
    7. Extent to which the research includes evaluation of relevant outcomes, which is in alignment with the overall performance measurement framework.
  4. Strength of team
    1. Strength and track record of the Network leadership in integrated multidisciplinary, patient-oriented research.
    2. Evidence of Network members with expertise in the wider translation of research into institutions, health care systems, policy, etc.
    3. Demonstration of the establishment of working partnerships that meaningfully integrate all key stakeholder groups including 1) researchers from different disciplines and regions across Canada and other countries (where appropriate); 2) patients/family representatives/informal care providers; 3) health policy makers; 4) health care professionals, with shared leadership, pooled resources, and equal commitment to success demonstrated by all key participants.
    4. Evidence that the Network has integrated the right people, resources, investments and organizations most needed to realize and sustain the proposed transformation so that it is positioned to influence the key levers of change.
    5. Evidence that the sex/gender champion is a core member of the Network with a track record and relevant expertise in the field to foster sex/gender integration across the activities of the Network.
  5. Patient-oriented research training and mentoring strategy
    1. Quality of the overall patient-oriented research training and mentoring strategy.
    2. Evidence that the strategy will be realized through an integrated multidisciplinary program that provides unique and innovative training and mentoring opportunities.
    3. Evidence that researchers and health care providers are actively engaged in training and mentoring opportunities, including a focus on patient engagement.
    4. Evidence that patients/family representatives/informal care providers are supported through training and mentoring to be active partners in the research.
    5. Evidence that policy decision makers are being supported through training and mentoring to ensure continuous engagement and active participation in research activities.
  6. Knowledge translation plan
    1. Quality of the knowledge translation plan, including its relevance and applicability to a wide scope of patients that encompasses sex and gender, ethnicity, cultural background, and socioeconomic class, and other end users (e.g. health service providers, health policy decision-makers, etc.).
    2. Feasibility of the knowledge translation plan, taking into consideration resources and time allocated to undertake identified knowledge translation activities.
    3. Feasibility and quality of the scale-up plan, taking into consideration approaches moving knowledge into the clinical research phase and knowledge translation and uptake mechanisms ultimately needed when moving the clinical research outcomes into clinical practice.
  7. Governance
    1. Appropriateness of the Network’s governance plan, including structure and functioning to achieve defined goals.
    2. Evidence of ultimate accountability; shared strategic leadership; transparency in decision-making; clearly defined roles, responsibilities and contributions, pooled intellectual, financial, and other resources; and other demonstrations that all key participants are highly committed to making transformative change happen.
    3. Clear identification of the roles and responsibilities in the governance structure of individuals representing 1) patients/family representatives/informal care providers; 2) health policy decision-makers; 3) researchers from different disciplines and regions across Canada and other countries (where appropriate); and 4) health care professionals.
    4. Clear identification of the roles and responsibilities of partners and, if applicable, a clear articulation of pooled resources.
    5. Appropriateness of the conflict resolution strategy.

* Note that the term patient includes patients, family and informal care-givers.

Please note that CIHR has implemented a requirement that all grant applicants respond to mandatory questions about whether sex and/or gender considerations are included in their grant proposals designs (effective December 2010). These questions are part of CIHR's wider strategy for meeting the requirements of a federal Health Portfolio Sex and Gender Bases Analysis (SGBA) Policy that requires all Health Portfolio organizations’ to apply SGBA in policy development, program design and organizational procedures. Anecdotal evidence indicates that applicants’ responses to the sex/gender questions are not consistently considered during peer review. The CIHR Sex and Gender Policy for grant applications is relevant to all areas of science and it is important that CIHR’s peer reviewers consider this policy when conducting proposal reviews. For more information on how to consider sex and/or gender in peer review, please see: Integrating Gender and Sex in Health Research: A Tool for CIHR Peer Reviewers Adherence to Sex and Gender Policy.

3.2 Budget Assessment

Reviewers will be required to assess to what extent the budget is appropriate and justified in relation to the proposed activities. Detailed scrutiny of each item will not be expected. Reviewers may recommend that the budget remain as requested or recommend an adjusted amount. If a reviewer adjusts the budget, he/she will be required to provide comments to justify their recommendation.

3.3 Issues to Flag

Reviewers will also be asked to flag applications that involve any of the issues outlined below. These issues are not to be considered as criteria for adjudication unless they impact the scientific quality of the proposal. For detailed regulations concerning these issues, please refer to the CIHR Funding Policies.

  1. Ethical and other policy requirements: Responsibility for ensuring that all research meets ethical standards is delegated to the local institution by CIHR. Ethics forms are not required as part of the application. However, the reviewer may flag specific issues, such as the use of human participants, animals, human tissues or hazardous material, or research that appears to involve Aboriginal people, if they feel they have not been adequately addressed.
  2. Human pluripotent stem cell research: Applications involving the use of human stem cells which will be funded will also be reviewed by the Stem Cell Oversight Committee (SCOC). Applicants are instructed to check the relevant box in the section entitled "Certification Requirements", but it is essential that this be reviewed by reviewers.

4. General Considerations

The success of the review system is critically dependent upon the willingness and ability of all reviewers to be fair and reasonable; to exercise rigorous judgment and to apply the same level of judgment for potential impact of the application; and to understand, and take into account in a balanced way, the particular context of each proposal. All reviewers must avoid conflict of interest and respect the confidentiality of the plans.

4.1 Confidentiality

Confidentiality is information about a person that shall not be disclosed directly or indirectly to anyone else without that person's prior expressed consent. The information provided by jurisdictions in their proposals is protected by the Privacy Act and is made available to external assessors for reviewing purposes only.

By law, applicants have access to their own proposal files. Therefore all written materials used in evaluating a proposal are made available to the jurisdictions when they are notified of a decision.

All materials related to the review process must be stored in a secure manner to prevent unauthorized access. Materials must be transmitted using secure carriers and technologies as per CIHR's guidelines on the handling of protected information. Any loss or theft of these materials must be reported to CIHR. When materials are no longer required, all material related to review must be destroyed using a secure method, or returned to CIHR for destruction.

4.2 Conflict of Interest

A conflict of interest is a conflict between a person's duties and responsibilities with regard to the review process, and that person's private, professional, business or public interests. Some factors to consider when determining if a conflict of interest exists are:

  1. Potential for professional or personal benefit;
  2. Level of leadership or authority;
  3. Professional or personal proximity to the competition or proposal being reviewed, or to an applicant;
  4. Direct or indirect financial interest in a competition or proposal being reviewed.

No panel member with a conflict of interest may participate in any part of the review of the proposal. A panel member is considered to have a conflict of interest with a proposal if he/she:

  • is from the same immediate department, institution, organization or company as the applicant, and who interacts with the applicant in the course of his/her duties at the department, institution, organization or company;
  • has collaborated, been a co-applicant or published with the applicant, within the last five years (exception will be made for CIHR funded Networks designed to increase partnerships among disciplines, institutions and thematic research);
  • has been a student or supervisor of the applicant within the last ten years;
  • is a close personal friend or relative of the applicant;
  • has had long-standing professional or personal differences with the applicant;
  • is in a position to gain or lose financially from the outcome of the proposal (e.g., holds stock in the company of an industry partner or a competitor); or
  • for some other reason feels that he/she cannot provide an objective review of the proposal.

All panel members (Reviewers, Virtual Chairs, etc.) are subject to the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations.

On behalf of CIHR, we would like to thank you for agreeing to serve as a peer review member. The success of the peer review process is made possible by dedicated people like you who generously give of their time and expertise, and your efforts are greatly appreciated by CIHR and the community.

Date modified: