CSR Advisory Council Discusses Future Directions for Peer Review

By: Yvette Seger
Wednesday, April 7, 2021

On March 29, the Advisory Council of the National Institutes of Health (NIH) Center for Scientific Review (CSR) convened to discuss the impact of the COVID-19 pandemic on peer review process and procedures.

In the Director’s report, CSR Director Noni Byrnes, PhD, presented findings from reviewer surveys to assess the impact of the virtual environment on the peer review process. In general, reviewers reported that their personal experience (e.g., ability to contribute to discussion and voice opinions) and overall impressions of the meeting were the same as in-person meetings. While reviewers did note a slight decrease of attention span in the past year of all-virtual meetings, key benefits of virtual meetings included the ability to recruit individuals previously unable to serve as reviewers, including clinicians and women, and an overall cost savings of $22 million in fiscal year 2020, which was returned to the NIH general operating budget. Byrnes noted that post-pandemic, CSR would likely pursue a hybrid approach for future peer review meetings, with study sections meeting in-person one or two times per year to build rapport and remaining meetings to be held virtually.

Council members also heard an update from the workgroup charged with simplifying review criteria for clinical trials. The issue of increasing complexity of grant applications and review criteria was initially raised in September 2019. Since then, two workgroups — one charged with examining review criteria for basic research project grants and the second charged with exploring review criteria for grants involving clinical trials — have convened and made recommendations to the CSR Advisory Council. Both groups agreed that the complexity of current review criteria ultimately reduced the quality of peer review.

To ensure peer review achieves its goal of identifying high-quality research proposals free from bias, both workgroups recommended streamlining review criteria to address three key factors: scientific importance, feasibility and rigor, and investigators and environment. Factor 1, “Importance of the Science,” would replace the current criterion of significance and address the question of whether the research should be done. The second factor, “Feasibility and Rigor,” would address two current criteria — approach and innovation — to assess the ability of the science to be done well. Finally, “Investigators and Environment,” would take a holistic approach to assessing the resources associated with a proposal. In addition to receiving scores on these three factors, each application would receive a score for “Overall Impact.”

While the proposed updates were well received by CSR Advisory Council members, it was noted that the recommendations would have to undergo several additional layers of review within NIH and the U.S. Department of Health and Human Services before changes could be implemented.