Selection Panel Review Guide – Strategy for Patient-Oriented Research (SPOR) Networks in Chronic Disease

Table of Contents

• 1. Highlights of the Funding Opportunity
  • 1.1 Background
  • 1.2 Objectives
  • 1.3 Focus of the Networks
  • 1.4 Funds Available
• 2. Role of the Stage 2 Selection Panel
• 3. Review Process
  • 3.1 Stage 1: Virtual Review (completed)
  • 3.2 Stage 2: Selection Panel Review
    • 3.2.1 Selection Panel Pre-Meeting Activities
    • 3.2.2 Selection Panel Meeting Process
• 4. General Considerations
  • 4.1 Confidentiality
  • 4.2 Conflict of Interest

1. Highlights of the Funding Opportunity

1.1 Background

Patient-oriented research, a cornerstone of evidence-informed health care, refers to a continuum of research, from initial studies in humans to comparative effectiveness and outcomes research, and the integration of this research into the health care system and clinical practice.

Canada’s Strategy for Patient-Oriented Research (SPOR) is a strategy that puts patients first. The vision of the strategy is that Canada will demonstrably improve health outcomes and enhance the health care experience for patients through the integration of evidence at all levels of the health care system.

Research and knowledge translation networks are one of the five core elements designed to realize the vision of SPOR. SPOR Networks are national, multi-institutional, multi-disciplinary and collaborative and are comprised of collaborations of patients, health professionals, decision makers, health researchers and other stakeholders. These networks are expected to develop important and effective relationships with the other core elements of SPOR and where appropriate, with other SPOR Networks.

This funding opportunity will provide support for the development and implementation of SPOR Networks in non-communicable chronic diseases. For the purpose of this opportunity, non-communicable chronic disease is defined as a long-term disease that is not passed from one person to another; it evolves over time, often progressing in severity. Although a non-communicable chronic disease may be prevented and can often be controlled, it is rarely cured.

1.2 Objectives

The primary objective of the opportunity will be to translate existing and new knowledge generated by basic biomedical, clinical, and population health research into testing of innovations that can improve clinical science and practice and foster policy changes, leading to transformative and measureable improvements in patient health outcomes, and in efficiency and effectiveness healthcare delivery within five years.

Network applicants are expected to show that the planned program of research and knowledge translation will have a significant impact on non-communicable chronic diseases and their co-morbidities by:

• Addressing patient-oriented research priorities and developing an interdisciplinary nation-wide agenda of outcomes-oriented research.
• Accelerating the translation of existing and new knowledge generated by basic biomedical, clinical, and population health research into clinical science and practice and policy applications.
• Strengthening the capacity to conduct patient-oriented research through an integrated multidisciplinary program that provides unique and innovative training and mentoring opportunities.
• Fostering the development and implementation of evidence-informed and gender-responsive practices, policies, services, products and programs to improve patient outcomes, access to care and quality, efficiency and effectiveness of health care.

1.3 Focus of the Networks

While Canada has a strong foundation on which to build, a number of factors impede the flow of information from basic biomedical research through clinical research to changes in patient care and improved health outcomes.

To address this issue, applicants are expected to:

• Focus on translating knowledge generated by basic biomedical, clinical and population health research into the clinical research phase where it will have significant, real-world impact. Note that moving what is gained from clinical research into policy and practice is within scope but should not be the principle focus of the proposal.
• Address first-in-human and beyond research in non-communicable chronic diseases that have resulted in areas of high disease burden.
• Include meaningful representation from each of the following stakeholder groups: 1) researchers; 2) health care professionals; 3) patients/family representatives/informal care providers; and 4) health policy makers.
• Demonstrate that it will be pan-Canadian in scope, where principal applicant(s) are from three of the following four regions:
  • East – includes Newfoundland and Labrador, Nova Scotia, Prince Edward Island and New Brunswick
  • Central – includes Quebec, Ontario and Manitoba
  • West – includes Saskatchewan, Alberta and British Columbia
  • North – includes Nunavut, Northwest Territories and Yukon

In addition, applicants are strongly encouraged to:

• Explore the opportunities for impact across multiple non-communicable chronic diseases.
• Include relevant international collaborations.

1.4 Funds Available

• $49.8M is available to fund four distinct networks.
• Each successful network will receive $12,450,000 over five years from CIHR and must be matched at a minimum of 1:1 with funding from non-federal sources.

2. Role of the Stage 2 Selection Panel

This Panel will be responsible for integrating the results of the virtual phase reviews and making recommendations on which applications in distinct areas should be funded.

3. Review Process

There is a two stage review process to adjudicate the applications: a virtual review of all applications followed by a face-to-face meeting of a Selection Panel, of which you are a member.

3.1 Stage 1: Virtual Review (completed)

Each application has been reviewed by six reviewers through a virtual process covering all key stakeholder groups, including patients, patient engagement experts, clinicians, researchers and policy experts. For each application assigned, reviewers were asked to provide an assessment of the strengths and weaknesses for each of the evaluation criteria outlined below and complete an “Evaluation Form” provided by CIHR.

As the virtual review phase unfolded, all six reviewers assigned to an application were invited to participate in a teleconference to discuss and understand discrepancies in reviews and budget recommendations (if applicable). Once the teleconference was complete, the reviewers were asked to finalize their reviews.

Evaluation Criteria

1. Vision, rationale, priorities and added value
   1. Coherence of the Network’s vision and rationale, with a focus on accelerating the translation of knowledge into clinical research and practice.
   2. Appropriateness of the patient*-driven priority-setting mechanism(s)/process(es) that will inform the program of research.
   3. Clarity of the expected outcomes and the added value the Network will bring in realizing these outcomes.
   4. Likelihood of impact on clinical practice within the timeframe of the Network funding.
   5. Clarity of the Network’s vision regarding analysis and expected outcomes as they relate to sex and gender, ethnicity, and other key social determinants of health, including geographic and economic.
   6. Evidence of explicit consideration being given to achieving equitable health impacts across patient/population sub-groups.
   7. Where applicable, the value-add of the cross-cutting nature of the proposal (i.e., the value-add of addressing multiple chronic diseases).
2. Feasibility, transformative impact and sustainability
   1. Creativity and innovativeness of the Network strategy, and the transformational potential for the clinical care of patients.
   2. Degree to which the proposal addresses chronic disease(s) that have resulted in areas of high disease burden and/or high healthcare costs to Canadian society.
   3. Demonstration that the Network is pan-Canadian in scope and design in such a way that it has the potential to significantly improve the lives of patients in three of the four designated regions in Canada: East, Central, West, North. Differential needs of patient populations in these geographic areas, and differential clinical service models are explicitly taken into account in the design of the Network.
   4. Quality and relevance of the stakeholder engagement strategy including attention to engagement of vulnerable populations.
   5. Quality and potential effectiveness of the Network’s sustainability plan.
   6. Quality of the partnerships and their potential to achieve outcomes and long-term sustainability.
   7. Evidence of leveraging existing resources and platforms, including SPOR SUPPORT Units.
   8. Relevance of self-assessed barriers and enablers to the Network strategy and appropriateness of the plan to address them, including a clear articulation of key risks and suitable mitigation strategies.
   9. Relevance of national synergies and, if applicable, international linkages.
   10. Quality of the performance measurement framework, including relevant metrics, to evaluate outcomes that matter most to patients, their healthcare providers, and health policy makers.
3. Patient-oriented research program
   1. Quality of the proposed research program and the extent to which it fits in the Network’s overall vision to address challenges and opportunities.
   2. Appropriateness of the research approach, i.e., the underlying hypotheses and/or propositions, research questions, design, research methods, as well as a realistic applicant assessment of limitations and alternatives.
   3. Evidence of appropriate and meaningful integration of patient engagement mechanisms to ensure that the program of research has been informed by patients from the onset and will continue to be informed by patients.
   4. Completeness of the literature review and its relevance to supporting the theoretical and/or conceptual framework for the proposed research program.
   5. The relevance of the research program for end users (e.g., patients, health service providers, health policy makers, etc.) and the extent to which the research program reflects patient priorities and is focused on improving patient outcomes.
   6. Quality and appropriateness of the strategy for addressing issues related to sex and gender, ethnicity, and social determinants of health, including geographic and economic. 
   7. Extent to which the research includes evaluation of relevant outcomes, which is in alignment with the overall performance measurement framework.
4. Strength of team
   1. Strength and track record of the Network leadership in integrated multidisciplinary, patient-oriented research.
   2. Evidence of Network members with expertise in the wider translation of research into institutions, health care systems, policy, etc.
   3. Demonstration of the establishment of working partnerships that meaningfully integrate all key stakeholder groups including 1) researchers from different disciplines and regions across Canada and other countries (where appropriate); 2) patients/family representatives/informal care providers; 3) health policy makers; 4) health care professionals, with shared leadership, pooled resources, and equal commitment to success demonstrated by all key participants.
   4. Evidence that the Network has integrated the right people, resources, investments and organizations most needed to realize and sustain the proposed transformation so that it is positioned to influence the key levers of change.
   5. Evidence that the sex/gender champion is a core member of the Network with a track record and relevant expertise in the field to foster sex/gender integration across the activities of the Network.
5. Patient-oriented research training and mentoring strategy
   1. Quality of the overall patient-oriented research training and mentoring strategy.
   2. Evidence that the strategy will be realized through an integrated multidisciplinary program that provides unique and innovative training and mentoring opportunities.
   3. Evidence that researchers and health care providers are actively engaged in training and mentoring opportunities, including a focus on patient engagement.
   4. Evidence that patients/family representatives/informal care providers are supported through training and mentoring to be active partners in the research.
   5. Evidence that policy decision makers are being supported through training and mentoring to ensure continuous engagement and active participation in research activities.
6. Knowledge translation plan
   1. Quality of the knowledge translation plan, including its relevance and applicability to a wide scope of patients that encompasses sex and gender, ethnicity, cultural background, and socioeconomic class, and other end users (e.g. health service providers, health policy decision-makers, etc.).
   2. Feasibility of the knowledge translation plan, taking into consideration resources and time allocated to undertake identified knowledge translation activities.
   3. Feasibility and quality of the scale-up plan, taking into consideration approaches moving knowledge into the clinical research phase and knowledge translation and uptake mechanisms ultimately needed when moving the clinical research outcomes into clinical practice.
7. Governance
   1. Appropriateness of the Network’s governance plan, including structure and functioning to achieve defined goals.
   2. Evidence of ultimate accountability; shared strategic leadership; transparency in decision-making; clearly defined roles, responsibilities and contributions, pooled intellectual, financial, and other resources; and other demonstrations that all key participants are highly committed to making transformative change happen.
   3. Clear identification of the roles and responsibilities in the governance structure of individuals representing 1) patients/family representatives/informal care providers; 2) health policy decision-makers; 3) researchers from different disciplines and regions across Canada and other countries (where appropriate); and 4) health care professionals.
   4. Clear identification of the roles and responsibilities of partners and, if applicable, a clear articulation of pooled resources.
   5. Appropriateness of the conflict resolution strategy.

* Note that the term patient includes patients, family and informal care-givers.

3.2 Stage 2: Selection Panel Review

The Selection Panel is responsible for calibrating the results of the virtual review process and providing CIHR with a recommendation on which applications in distinct areas should be funded. The Panel will consist of researchers, clinicians, policy makers, patient engagement experts and patients. Each application will be assigned reviewers representing these expertise areas.

3.2.1 Selection Panel Pre-Meeting Activities

Prior to the meeting, each Selection Panel member will be assigned a number of applications and each application will be assigned to six Selection Panel members. For each application, the Selection Panel member will have access to information from the virtual review stage, including the reviews, the Summary of Research Proposal and the full application.

Members are asked to rank their assigned applications prior to the meeting. CIHR will prepare a consolidated ranking list with all of the applications to be presented at the meeting.

3.2.2 Selection Panel Meeting Process

Calibration

In order to ensure that the virtual reviews are properly calibrated, to start the meeting, the Selection Panel members will discuss one from each of the three following groups: 1) the bottom group, with the lowest combined ranking, 2) the middle group (the 8th ranked application) and 3) the top group, with the highest combined ranking.

Discussion

The panel discussion will focus on how proposals respond to the evaluation criteria and the strategic alignment with the funding opportunity objectives and focus areas.

Approach - Discussion Procedure (sequence of steps):

1. Any Selection Panel member in conflict with the application for discussion is asked to leave the meeting room for the discussion.
2. The six assigned reviewers will provide the key strengths and weaknesses of the application, and highlight the key points leading to their assessment. The member starting the discussion is asked to provide a brief overview of the application.
3. The other members assigned to the application are asked to provide additional comments, focusing on points of agreement or disagreement, and elaborating on aspects not already addressed. The other Panel members are then invited to the discussion (e.g., additional comments and questions to clarify points of disagreement). The Chair moderates the discussion.
4. The Scientific Officer (SO) will take notes of the discussion as it proceeds and will share them with the Panel at the conclusion of the review of each application making additions and modifications as advised.
5. The Panel will deliberate extensively on all applications.
6. The Panel will vote privately to determine the ranking list using real-time technology.

Confirmation of the Descriptors

In addition to the ranking list, the panel will discuss and decide on a descriptor for each application.

Intended Outcome: The intended outcome is to have a descriptor for each application and a final ranking list from which the panel can recommend applications in distinct areas for funding.

Panel Determination of Distinct Focus Areas: Once a ranking list is determined, the panel will discuss which of the applications are in distinct areas and therefore should be recommended for funding.

4. General Considerations

4.1 Confidentiality

Confidentiality is information about a person that shall not be disclosed directly or indirectly to anyone else without that person's prior expressed consent. The information provided by jurisdictions in their proposals is protected by the Privacy Act and is made available to external assessors for reviewing purposes only.

By law, applicants have access to their own proposal files. Therefore all written materials used in evaluating a proposal are made available to the jurisdictions when they are notified of a decision.

All materials related to the review process must be stored in a secure manner to prevent unauthorized access. Materials must be transmitted using secure carriers and technologies as per CIHR's guidelines on the handling of protected information. Any loss or theft of these materials must be reported to CIHR. When materials are no longer required, all material related to review must be destroyed using a secure method, or returned to CIHR for destruction.

4.2 Conflict of Interest

A conflict of interest is a conflict between a person's duties and responsibilities with regard to the review process, and that person's private, professional, business or public interests. Some factors to consider when determining if a conflict of interest exists are:

1. Potential for professional or personal benefit;
2. Level of leadership or authority;
3. Professional or personal proximity to the competition or proposal being reviewed, or to an applicant;
4. Direct or indirect financial interest in a competition or proposal being reviewed.

No panel member with a conflict of interest may influence any part of the review of the proposal. A panel member is considered to have a conflict of interest with a proposal if he/she:

• is from the same immediate department, institution, organization or company as the applicant, and who interacts with the applicant in the course of his/her duties at the department, institution, organization or company;
• has collaborated, been a co-applicant or published with the applicant, within the last five years (exception will be made for CIHR funded Networks designed to increase partnerships among disciplines, institutions and thematic research);
• has been a student or supervisor of the applicant within the last ten years;
• is a close personal friend or relative of the applicant;
• has had long-standing professional or personal differences with the applicant;
• is in a position to gain or lose financially from the outcome of the proposal (e.g., holds stock in the company of an industry partner or a competitor); or
• for some other reason feels that he/she cannot provide an objective review of the proposal.

All panel members are subject to the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations.

Date modified:

2016-02-25