Introductory remarks by CIHR President, Dr. Alain Beaudet
Peer Review Working Meeting – July 13, 2016 in Ottawa

Working Meeting of the Health Research Community

(Check against delivery)

  • Many thanks to all of you for taking the time to participate in today's critical meeting, despite the short notice. 
  • I want to stress the importance of your presence here today for helping us move forward. We at CIHR have heard your concerns loud and clear, and we are here to develop with you a set of pragmatic solutions for improving the peer review of investigator-initiated grants.
  • Solutions that can be implemented in the next round of competitions which is about to be launched – hence the urgency of this meeting.
  • Our hope is to be able to course correct some of the most controversial changes to peer review, without losing the overall objectives of the reforms. To improve on where we are, and to achieve a system that you approve of and have full confidence in.

The online review system

  • As you know, a major element of the changes to our peer review system has been the move towards a personalized – or application focussed – evaluation of proposals by a set of five experts. 
  • This was in part a response to the recommendations of CIHR's last International Review Panel, in addition to a call by the scientific community – highlighted in a 2010 IPSOS Reid poll – that we fix a peer review system that was perceived as lacking 'quality and consistency'.
  • CIHR was also facing increasing difficulty in force fitting applications into standing committees. Indeed, we had reached a point where over 30% of the applications received, many of them interdisciplinary, were being turned down – for lack of expertise – by the panel to which they were first submitted.
  • The theoretical advantage of a new online system was twofold:
    1. to ensure that all aspects of any application – and particularly of interdisciplinary ones – would benefit from appropriate, world-wide expertise, while minimizing conflicts of interest; and
    2. to allow us to pursue excellence irrespective of the field of research.
  • The online reviews were never meant to totally replace the work of face-to-face committees, but rather to increase the quality and stringency of the screening process.
  • In other words, the aim was to expand and improve a system that was triaging over 50% of the grants based on two reviews, into one that would help us focus on the most contentious ones, based on five reviews.
  • We also wanted in so doing to facilitate the task and decrease the work load of face-to-face panels.
  • However, this was the theory. The implementation proved considerably more challenging.
  • There were, as you well know, problems with the delivery of the new process, particularly in this first project scheme pilot.  Frankly, the gap between theory and practice proved greater than anticipated.
  • There was a tipping point in the process – this being the alarming increase in reports of poor quality on line reviews and lack of appropriate online discussions.
  • This caused real and significant concern to me - and to the Minister of Health.
  • I can only assure you that we did everything we possibly could to correct the situation as soon as we became aware that it was problematic.
  • For instance, we ensured that any application which did not benefit from a minimum of three online reviews, or from proper online discussion, would be sent for face-to-face appraisal.
  • Also, I have asked for a full external assessment of the quality of the more than 16 000 reviews that we received in the course of the first project scheme competition, and I commit to make the results of this audit public.
  • This being said, CIHR cannot afford to lose the confidence of the scientific community.
  • The peer review system will only work if the community is on side. You are, after all, not only the beneficiaries, but also the actors – as reviewers – in this process. We need to be in this together.

Going forward

  • I believe passionately about the need for adapting our adjudication processes to the times and the changing nature of science. But I also admit that we haven't yet perfected the system to get there.
  • I am convinced however that we can work together towards developing a way forward that will allow the equitable review of the most meritorious proposals across the entire spectrum of health research: from Pillar 1 to pillar 4, from disciplinary to multi-disciplinary, from fundamental to applied.
  • One thing that I believe is critical is that we ensure that irrespective of the system we arrive at today, it is structured in a way that allows data collection so that our ultimate course of action may be evidence-based.
  • It is also important that we submit the system to external eyes – external to Canada I mean – to look at this issue dispassionately, and give us some advice. In this regard, you may recall that CIHR has already committed to commissioning an independent evaluation of our Peer Review systems by an international panel of experts (distinct from the Naylor panel whose mandate is considerably broader).
  • I am very much looking forward to hearing your ideas and advice today and as we move ahead together.
  • Thank you.
Date modified: