‘Are you siding with a personality or the grant proposal?’: observations on how peer review panels function

Research Integrity and Peer Review 2 (1) (2017)
  Copy   BIBTEX


BackgroundIn Australia, the peer review process for competitive funding is usually conducted by a peer review group in conjunction with prior assessment from external assessors. This process is quite mysterious to those outside it. The purpose of this research was to throw light on grant review panels (sometimes called the ‘black box’) through an examination of the impact of panel procedures, panel composition and panel dynamics on the decision-making in the grant review process. A further purpose was to compare experience of a simplified review process with more conventional processes used in assessing grant proposals in Australia.MethodsThis project was one aspect of a larger study into the costs and benefits of a simplified peer review process. The Queensland University of Technology (QUT)-simplified process was compared with the National Health and Medical Research Council’s (NHMRC) more complex process. Grant review panellists involved in both processes were interviewed about their experience of the decision-making process that assesses the excellence of an application. All interviews were recorded and transcribed. Each transcription was de-identified and returned to the respondent for review. Final transcripts were read repeatedly and coded, and similar codes were amalgamated into categories that were used to build themes. Final themes were shared with the research team for feedback.ResultsTwo major themes arose from the research: (1) assessing grant proposals and (2) factors influencing the fairness, integrity and objectivity of review. Issues such as the quality of writing in a grant proposal, comparison of the two review methods, the purpose and use of the rebuttal, assessing the financial value of funded projects, the importance of the experience of the panel membership and the role of track record and the impact of group dynamics on the review process were all discussed. The research also examined the influence of research culture on decision-making in grant review panels. One of the aims of this study was to compare a simplified review process with more conventional processes. Generally, participants were supportive of the simplified process.ConclusionsTransparency in the grant review process will result in better appreciation of the outcome. Despite the provision of clear guidelines for peer review, reviewing processes are likely to be subjective to the extent that different reviewers apply different rules. The peer review process will come under more scrutiny as funding for research becomes even more competitive. There is justification for further research on the process, especially of a kind that taps more deeply into the ‘black box’ of peer review.



    Upload a copy of this work     Papers currently archived: 79,912

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Commensuration Bias in Peer Review.Carole J. Lee - 2015 - Philosophy of Science 82 (5):1272-1283,.
Peer review and innovation.Raymond Spier - 2002 - Science and Engineering Ethics 8 (1):99-108.
The ethics of peer review in bioethics.David Wendler & Franklin Miller - 2014 - Journal of Medical Ethics 40 (10):697-701.
The Necessity of Commensuration Bias in Grant Peer Review.Remco Heesen - 2021 - Ergo: An Open Access Journal of Philosophy 8 (39):423--443.
The principles and practices of Peer review.Ronald N. Kostoff - 1997 - Science and Engineering Ethics 3 (1):19-34.


Added to PP

6 (#1,128,408)

6 months
1 (#479,283)

Historical graph of downloads
How can I increase my downloads?

Citations of this work

No citations found.

Add more citations

References found in this work

No references found.

Add more references