Evaluating the Pros and Cons of Different Peer Review Policies via Simulation

Science and Engineering Ethics 22 (4):1073-1094 (2016)
  Copy   BIBTEX

Abstract

In the academic world, peer review is one of the major processes in evaluating a scholars contribution. In this study, we are interested in quantifying the merits of different policies in a peer review process, such as single-blind review, double-blind review, and obtaining authors feedback. Currently, insufficient work has been undertaken to evaluate the benefits of different peer review policies. One of the major reasons for this situation is the inability to conduct any empirical study because data are presently unavailable. In this case, a computer simulation is one of the best ways to conduct a study. We perform a series of simulations to study the effects of different policies on a peer review process. In this study, we focus on the peer review process of a typical computer science conference. Our results point to the crucial role of program chairs in determining the quality and diversity of the articles to be accepted for publication. We demonstrate the importance of discussion among reviewers, suggest circumstances in which the double-blind review policy should be adopted, and question the credibility of the authors feedback mechanism. Finally, we stress that randomness plays an important role in the peer review process, and this role cannot be eliminated. Although our model may not capture every component of a peer review process, it covers some of the most essential elements. Thus, even the simulation results clearly cannot be taken as literal descriptions of an actual peer review process. However, we can at least still use them to identify alternative directions for future study.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 92,369

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

A Place for Cost-Benefit Analysis.David Schmidtz - 2001 - Noûs 35 (s1):148 - 171.
The pros and cons of having a word for it.S. F. Walker - 1983 - Behavioral and Brain Sciences 6 (1):156-157.
Ifs and Cans: Pros and Cons.Douglas Walton - 1975 - Pacific Philosophical Quarterly 56 (3):242.
The Creation of Clones: Pros and Cons?Huang-Chieh Yu - 2003 - Philosophy and Culture 30 (2):182-187.
The Pros and Cons of Human Cloning.Martin LaBar - 1984 - Thought: Fordham University Quarterly 59 (3):319-333.
The pros and cons of masked priming.Kenneth Forster - 1998 - Journal Of Psycholinguistic Research 27 (2):203-233.
Globalisation.D. P. Chattopadhyaya - 2007 - The Proceedings of the Twenty-First World Congress of Philosophy 13:63-71.
Peer Review: A Critical Inquiry.David Shatz - 2004 - Rowman & Littlefield Publishers.

Analytics

Added to PP
2015-09-03

Downloads
25 (#637,843)

6 months
9 (#319,342)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Citations of this work

No citations found.

Add more citations

References found in this work

Social Choice and Individual Values.Irving M. Copi - 1952 - Science and Society 16 (2):181-181.
Social Choice and Individual Values.Kenneth Joseph Arrow - 1951 - New York, NY, USA: Wiley: New York.
Bias in Peer Review.Carole J. Lee, Cassidy R. Sugimoto, Guo Zhang & Blaise Cronin - 2013 - Journal of the American Society for Information Science and Technology 64 (1):2-17.

View all 10 references / Add more references