Peer-review has become increasingly important to the way scholarly journals assess whether a manuscript is suitable for publication [...].

This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings (...) such as invited papers, early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)

While much literature has sprouted on peer review, this is the first book-length, wide-ranging study that utilizes methods and resources of contemporary philosophy. It covers the tension between peer review and the liberal notion that truth emerges when ideas proliferate in the marketplace of ideas; arguments for and against blind review of submissions; the alleged conservatism of peer review; the anomalous nature of book reviewing; the status of non-peer-reviewed publications; and the (...) future of peer review. (shrink)

BackgroundThe aim of this survey was to determine the level of awareness and understanding of peer review and peer review models amongst junior hospital doctors and whether this influences clinical decision-making.MethodsA 30-question online anonymous survey was developed aimed at determining awareness of peer review models and the purpose of peer review, perceived trustworthiness of different peer review models and the role of peer review in clinical decision-making. It was (...) sent to 800 trainee doctors in medical specialties on the University College London Partners trainee database.ResultsThe response rate was (178/800) 22%. Most respondents were specialist registrars. Checking that research is conducted correctly (152/178, 85%) and the data interpreted correctly (148/178, 83%) were viewed as the most important purposes of peer review. Most respondents were aware of open (133/178, 75%), double-blind (125/178, 70%) and single-blind peer review (121/178, 68%). 101/178 (57%) had heard of collaborative, 87/178 (49%) of post publication and 29/178 (16%) of decoupled peer review. Of those who were aware of double-blind, single-blind open and collaborative peer review, 85 (68%), 82 (68%), 74 (56%) and 24 (24%), respectively, understood how they worked. The NEJM, Lancet and The BMJ were deemed to have most trustworthy peer review, 137/178 (77%), 129/178 (72%) and 115/178 (65%), respectively. That peer review had taken place was important for a journal content to be used for clinical decision-making 152/178 (85%), but the ability to see peer review reports was not as important 22/178 (12%). Most felt there was a need for peer review training and that this should be at the specialist registrar stage of training.ConclusionsJunior hospital doctors view peer review to be important as a means of quality control, but do not value the ability to scrutinize peer review themselves. The unquestioning acceptance of peer review as final validation in the field of medicine emphasises not only the responsibility held by medical journals to ensure peer review is done well but also the need to raise awareness amongst the medical community of the limitations of the current peer review process. (shrink)

BackgroundThere is increasing need for peer reviewers as the scientific literature grows. Formal education in biostatistics and research methodology during residency training is lacking. In this pilot study, we addressed these issues by evaluating a novel method of teaching residents about biostatistics and research methodology using peer review of standardized manuscripts. We hypothesized that mentored peer review would improve resident knowledge and perception of these concepts more than non-mentored peer review, while improving (...) class='Hi'>review quality.MethodsA partially blinded, randomized, controlled multi-center study was performed. Seventy-eight neurology residents from nine US neurology programs were randomized to receive mentoring from a local faculty member or not. Within a year, residents reviewed a baseline manuscript and four subsequent manuscripts, all with introduced errors designed to teach fundamental review concepts. In the mentored group, mentors discussed completed reviews with residents. Primary outcome measure was change in knowledge score between pre- and post-tests, measuring epidemiology and biostatistics knowledge. Secondary outcome measures included level of confidence in the use and interpretation of statistical concepts before and after intervention, and RQI score for baseline and final manuscripts.ResultsSixty-four residents (82%) completed initial review with gradual decline in completion on subsequent reviews. Change in primary outcome, the difference between pre- and post-test knowledge scores, did not differ between mentored (−8.5%) and non-mentored (−13.9%) residents (p = 0.48). Significant differences in secondary outcomes (using 5-point Likert scale, 5 = strongly agree) included mentored residents reporting enhanced understanding of research methodology (3.69 vs 2.61; p = 0.001), understanding of manuscripts (3.73 vs 2.87; p = 0.006), and application of study results to clinical practice (3.65 vs 2.78; p = 0.005) compared to non-mentored residents. There was no difference between groups in level of interest in peer review (3.00 vs 3.09; p = 0.72) or the quality of manuscript review assessed by the Review Quality Instrument (RQI) (3.25 vs 3.06; p = 0.50).ConclusionsWe used mentored peer review of standardized manuscripts to teach biostatistics and research methodology and introduce the peer review process to residents. Though knowledge level did not change, mentored residents had enhanced perception in their abilities to understand research methodology and manuscripts and apply study results to clinical practice. (shrink)

Peer review is a topic of considerable concern to many researchers, and there is a correspondingly large body of research on the topic. This issue of Science and Engineering Ethics presents recent work on peer review that is both grounded in empirical science and is applicable to policy decisions. This research raises two basic questions; (a) how does current peer review operate, and (b) how can it be improved? Topics addressed include descriptions of how (...) peer review is used in Federal agencies. whether peer review leads to better manuscripts, demographic characteristics of authors or reviewers (status or institutional affiliation), blinding of reviewers, authors, or results, reliability and consistency of reviews, accepting a paper before the study is done, simultaneous submission, and use of dispute resolution procedures such as scientific dialectical and pleading protocols. (shrink)

The double‐blind peer‐review of manuscripts for potential publication is a longstanding tradition in the production of scholarship. Nursing has adopted this tradition to secure a place of legitimacy and authority for its scholarship amongst the other disciplines in the academy. However, despite its ubiquity and avowed utility, the peer‐review has not generally been the subject of much research let alone intense philosophical scrutiny and debate. This manuscript attempts such an engagement with a view to uncovering (...) specific concerns about the essentially conservative and sometimes restrictive effects the double‐blind peer‐review produces. Drawing on the deconstructionist writings of Derrida and his acolytes this paper attempts to dig beneath the surface mechanics of the double‐blind peer‐review and in so doing, expose its rather shaky philosophical foundations. It is written to open debate from others who too, have harboured doubts about its adequacy and supremacy as a technology in the production of (legitimate) knowledge. (shrink)

A good deal has been written on the ethics of peer review, especially in the scientific and medical literatures. In contrast, we are unaware of any articles on the ethics of peer review in bioethics. Recognising this gap, we evaluate the extant proposals regarding ethical standards for peer review in general and consider how they apply to bioethics. We argue that scholars have an obligation to perform peer review based on the extent (...) to which they personally benefit from the peer review process. We also argue, contrary to existing proposals and guidelines, that it can be appropriate for peer reviewers to benefit in their own scholarship from the manuscripts they review. With respect to bioethics in particular, we endorse double-blind review and suggest several ways in which the peer review process might be improved. (shrink)

A growing body of literature has identified potential problems that can compromise the quality, fairness, and integrity of journal peer review, including inadequate review, inconsistent reviewer reports, reviewer biases, and ethical transgressions by reviewers. We examine the evidence concerning these problems and discuss proposed reforms, including double-blind and open review. Regardless of the outcome of additional research or attempts at reforming the system, it is clear that editors are the linchpin of peer review, (...) since they make decisions that have a significant impact on the process and its outcome. We consider some of the steps editors should take to promote quality, fairness and integrity in different stages of the peer review process and make some recommendations for editorial conduct and decision-making. (shrink)

Scientific editors’ policies, including peer review, are based mainly on tradition and belief. Do they actually achieve their desired effects, the selection of the best manuscripts and improvement of those published? Editorial decisions have important consequences—to investigators, the scientific community, and all who might benefit from correct information or be harmed by misleading research results. These decisions should be judged not just by intentions of reviewers and editors but also by the actual consequences of their actions. A small (...) but growing number of studies has put editorial policies to a strong scientific test. In a randomized, controlled trial. blinding reviewers to author and institution was usually successful and improved the quality of reviews. Two studies have shown that, contrary to conventional wisdom, reviewers early in their careers give better reviews than senior reviewers. Many studies have shown low agreement between reviewers but there is disagreement about whether this represents a failing of peer review or the expected and valuable effect of choosing reviewers with complementary expertise. In a study of whether manuscripts are improved by peer review and editing, articles published in Annals of Internal Medicine were improved in 33 of 34 dimensions of reporting quality, but published articles still had room for improvement. Because of the central place of peer review in the scientific community and the resources it requires, more studies are needed to define what it does and does not accomplish. (shrink)

Scientific editors’ policies, including peer review, are based mainly on tradition and belief. Do they actually achieve their desired effects, the selection of the best manuscripts and improvement of those published? Editorial decisions have important consequences—to investigators, the scientific community, and all who might benefit from correct information or be harmed by misleading research results. These decisions should be judged not just by intentions of reviewers and editors but also by the actual consequences of their actions. A small (...) but growing number of studies has put editorial policies to a strong scientific test. In a randomized, controlled trial. blinding reviewers to author and institution was usually successful and improved the quality of reviews. Two studies have shown that, contrary to conventional wisdom, reviewers early in their careers give better reviews than senior reviewers. Many studies have shown low agreement between reviewers but there is disagreement about whether this represents a failing of peer review or the expected and valuable effect of choosing reviewers with complementary expertise. In a study of whether manuscripts are improved by peer review and editing, articles published in Annals of Internal Medicine were improved in 33 of 34 dimensions of reporting quality, but published articles still had room for improvement. Because of the central place of peer review in the scientific community and the resources it requires, more studies are needed to define what it does and does not accomplish. (shrink)

In the academic world, peer review is one of the major processes in evaluating a scholars contribution. In this study, we are interested in quantifying the merits of different policies in a peer review process, such as single-blind review, double-blind review, and obtaining authors feedback. Currently, insufficient work has been undertaken to evaluate the benefits of different peer review policies. One of the major reasons for this situation is the inability (...) to conduct any empirical study because data are presently unavailable. In this case, a computer simulation is one of the best ways to conduct a study. We perform a series of simulations to study the effects of different policies on a peer review process. In this study, we focus on the peer review process of a typical computer science conference. Our results point to the crucial role of program chairs in determining the quality and diversity of the articles to be accepted for publication. We demonstrate the importance of discussion among reviewers, suggest circumstances in which the double-blind review policy should be adopted, and question the credibility of the authors feedback mechanism. Finally, we stress that randomness plays an important role in the peer review process, and this role cannot be eliminated. Although our model may not capture every component of a peer review process, it covers some of the most essential elements. Thus, even the simulation results clearly cannot be taken as literal descriptions of an actual peer review process. However, we can at least still use them to identify alternative directions for future study. (shrink)

This paper examines the problem of vituperative feedback from peer reviewers. We argue that such feedback is morally unacceptable, insofar as it humiliates authors and damages their dignity. We draw from social-psychological research to explore those aspects of the peer-review process in general and the anonymity of blind reviewing in particular that contribute to reviewers’ humiliating comments. We then apply Iris Murdoch's ideas about a virtuous consciousness and humility to make the case that peer referees (...) have a moral obligation not to humiliate the authors whose work they review. (shrink)

Following publication of this article [1] it was brought to our attention that we omitted to provide credit for Table 1. While the content of the table and the systematization of blinding in review have been referenced in the text as coming from [2], the credit line for Table 1 should have been added as follows: “Reproduced with permission from [2] licensed under a CC BY-NC-ND 3.0 License”. The original publication of this article has been corrected accordingly.

High percentages of submitted papers are rejected at editorial levels without offering a second chance to authors by sending their papers for further peer-reviews. In most cases, the rejections are typical quick answers without helpful argumentations related to the content of the rejected material. More surprisingly, some journals vaunt their high rejection rates as a “mark of prestige”!However, journals that reject high percentages of submitted papers have built their prominent positions based on a flawed measure, the impact factor, and (...) from a long and favorable historical context. Their shareholders may think that they are allowed to have a large margin of rejection rates without affecting their sponsorship or funding sources thanks to an extended anchorage since tens, or in some cases hundreds, of years compared to unknown or new journals that struggle to pave a way in the scientific publication world. Historical anchorage of some journals also makes it unfair to compare old and new .. (shrink)

The _Maimonides Review_ is an annual collection of double-blind peer-reviewed articles, which seeks to provide a broad international arena for an intellectual exchange of ideas between the disciplines of philosophy, theology, religion, cultural history, and literature and to showcase their multifarious junctures within the framework of Jewish studies.

In 2014 Jennifer Cramblett, a white lesbian, filed a Complaint for Wrongful Birth alleging that the Midwest Sperm Bank mistakenly provided sperm from an African–American donor. In this article, we trace the complex and overlapping lines of legal and social inheritance that have conditioned not only the possibility of such a lawsuit, but also the legal language and arguments within the Complaint itself. First, we trace the racial politics of homonormativity, which set the conditions of possibility for an out, white (...) lesbian to bring this case forward. Second, we trace the inheritance of wrongful birth tort law, reviewing its prior race and disability-related uses, and its basis in feminist reproductive rights. Third, we trace how disability, race and sexuality interlock within the eugenic inheritance of both ‘wrongful birth’ and reproductive technologies. Finally, we follow traces of racial inheritance, namely, the loss of white property and proximity to whiteness. (shrink)

The _Maimonides Review_ is an annual collection of double-blind peer-reviewed articles, which seeks to provide a broad international arena for an intellectual exchange of ideas between the disciplines of philosophy, theology, religion, cultural history, and literature and to showcase their multifarious junctures within the framework of Jewish studies.

The Maimonides Review of Philosophy and Religion is an annual collection of double-blind peer-reviewed articles that seeks to provide a broad international arena for an intellectual exchange of ideas between the disciplines of philosophy, theology, religion, cultural history, and literature and to showcase their multifarious junctures within the framework of Jewish studies. Contributions to the Review place special thematic emphasis on scepticism within Jewish thought and its links to other religious traditions and secular worldviews. The (...) class='Hi'>Review is interested in the tension at the heart of matters of reason and faith, rationalism and mysticism, theory and practice, narrativity and normativity, doubt and dogma. (shrink)

Contemporary debates about scientific institutions and practice feature many proposed reforms. Most of these require increased efforts from scientists. But how do scientists’ incentives for effort interact? How can scientific institutions encourage scientists to invest effort in research? We explore these questions using a game-theoretic model of publication markets. We employ a base game between authors and reviewers, before assessing some of its tendencies by means of analysis and simulations. We compare how the effort expenditures of these groups interact in (...) our model under a variety of settings, such as double-blind and open review systems. We make a number of findings, including that open review can increase the effort of authors in a range of circumstances and that these effects can manifest in a policy-relevant period of time. However, we find that open review’s impact on authors’ efforts is sensitive to the strength of several other influences. (shrink)

PurposeIf an individual has been blind since birth due to a treatable eye condition, ocular treatment is urgent. Even a brief period of visual deprivation can alter the development of the visual system. The goal of our structured scoping review was to understand how we might better support children with delayed access to ocular treatment for blinding conditions.MethodWe searched MEDLINE, Embase and Global Health for peer-reviewed publications that described the impact of early and extended bilateral visual deprivation.ResultsOf (...) 551 reports independently screened by two authors, 42 studies met our inclusion criteria. Synthesizing extracted data revealed several trends. The data suggests persistent deficits in visual acuity, contrast sensitivity, global motion, and visual-motor integration, and suspected concerns for understanding complex objects and faces. There is evidence for resilience in color perception, understanding of simple shapes, discriminating between a face and non-face, and the perception of biological motion. There is currently insufficient data about specific habilitation strategies to update low vision services, but there are several insights to guide future research in this domain.ConclusionThis summary will help guide the research and services provision to help children learn to see after early and extended blindness. (shrink)

This letter contains suggestions for editors and software providers to help avoid affiliation bias in the initial and concluding stages of the peer review process. Submission management systems have a responsibility to ensure protection against affiliation bias. This can be achieved by automatically withholding the author’s identity and affiliation information from all editors, including the Editor-in-Chief, until a decision about publication has been made. Journals relying on email-based submissions are in a more difficult situation. Not having external support (...) in confirming the submitting author’s identity, this dilemma can be resolved through the implementation of strict instructions and guidelines concerning manuscript submission and handling. In this case the “human factor” cannot be fully eliminated, while automated processes can only be changed intentionally and with a certain delay. Therefore, with the suggested steps, submission management systems could become a strong guard against unconscious bias while also making conscious bias more difficult to impact the peer review process. (shrink)

Rooted in altruism theory, the purpose of the double-blind academic journal peer-review process is to: assess the quality of scientific research, minimize the potential for nepotism, and; advance the standards of research through high-quality, constructive feedback. However, considering the limited, if any, public recognition and monetary incentives that referees receive for reviewing manuscripts, academics are often reluctant to squander their limited time toward peer reviewing manuscripts. If they do accept such invitations, referees, at times, do not (...) invest the appropriate time needed and, as a result, scantily review manuscripts, which adversely affects the quality of the review. In addition, given that authors’ identities are not blind to journal editors, there is the potential for bias toward well-established academics from highly-ranked institutions. As a result of these issues, the aims of the academic journal review process are currently not being fulfilled. To rectify these issues, several recommendations, namely: single-blind the editors, pay reviewers, standardize the review process, increase the acceptance standards at academic conferences, and provide constructive feedback, are offered. (shrink)

BackgroundDouble-blind peer review has been proposed as a possible solution to avoid implicit referee bias in academic publishing. The aims of this study are to analyse the demographics of corresponding authors choosing double-blind peer review and to identify differences in the editorial outcome of manuscripts depending on their review model.MethodsData includes 128,454 manuscripts received between March 2015 and February 2017 by 25 Nature-branded journals. We investigated the uptake of double-blind review in (...) relation to journal tier, as well as gender, country, and institutional prestige of the corresponding author. We then studied the manuscripts’ editorial outcome in relation to review model and author’s characteristics. The gender (male, female, or NA) of the corresponding authors was determined from their first name using a third-party service (Gender API). The prestige of the corresponding author’s institutions was measured from the data of the Global Research Identifier Database (GRID) by dividing institutions in three prestige groups with reference to the 2016 Times Higher Education (THE) ranking. We employed descriptive statistics for data exploration, and we tested our hypotheses using Pearson’s chi-square and binomial tests. We also performed logistic regression modelling with author update, out-to-review, and acceptance as response, and journal tier, author gender, author country, and institution as predictors.ResultsAuthor uptake for double-blind submissions was 12% (12,631 out of 106,373). We found a small but significant association between journal tier and review type (p value < 0.001, Cramer’s V = 0.054, df = 2). We had gender information for 50,533 corresponding authors and found no statistically significant difference in the distribution of peer review model between males and females (p value = 0.6179). We had 58,920 records with normalised institutions and a THE rank, and we found that corresponding authors from the less prestigious institutions are more likely to choose double-blind review (p value < 0.001, df = 2, Cramer’s V = 0.106). In the ten countries with the highest number of submissions, we found a large significant association between country and review type (p value < 0.001, df = 10, Cramer’s V = 0.189). The outcome both at first decision and post review is significantly more negative (i.e. a higher likelihood for rejection) for double-blind than single-blind papers (p value < 0.001, df = 1, Cramer’s V = 0.112 for first decision; p value < 0.001; df = 1, Cramer’s V = 0.082 for post-review decision).ConclusionsThe proportion of authors that choose double-blind review is higher when they submit to more prestigious journals, they are affiliated with less prestigious institutions, or they are from specific countries; the double-blind option is also linked to less successful editorial outcomes. (shrink)

The _Maimonides Review of Philosophy and Religion_ is an annual collection of double-blind peer-reviewed articles, which seeks to provide a broad international arena for an intellectual exchange of ideas between the disciplines of philosophy, theology, religion, cultural history, and literature and to showcase their multifarious junctures within the framework of Jewish studies.

The loss or absence of vision is probably one of the most incapacitating events that can befall a human being. The importance of vision for humans is also reflected in brain anatomy as approximately one third of the human brain is devoted to vision. It is therefore unsurprising that throughout history many attempts have been undertaken to develop devices aiming at substituting for a missing visual capacity. In this review, we present two concepts that have been prevalent over the (...) last two decades. The first concept is sensory substitution, which refers to the use of another sensory modality to perform a task that is normally primarily sub-served by the lost sense. The second concept is cross-modal plasticity, which occurs when loss of input in one sensory modality leads to reorganization in brain representation of other sensory modalities. Both phenomena are training-dependent. We also briefly describe the history of blindness from ancient times to modernity, and then proceed to address themeansthat have been used to help blind individuals, with an emphasis on modern technologies, invasive (various type of surgical implants) and non-invasive devices. With the advent of brain imaging, it has become possible to peer into the neural substrates of sensory substitution and highlight the magnitude of the plastic processes that lead to a rewired brain. Finally, we will address the important question of the value and practicality of the available technologies and future directions. (shrink)

Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are (...) either positive or neutral. We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we (...) selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated. (shrink)

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.

Peer review and publication is one of the factors proposed in Daubert v. Merrell Dow Pharmaceuticals, Inc. as indicia of the reliability of scientific testimony. This Article traces the origins of the peer-review system, the process by which it became standard at scientific and medical journals, and the many roles it now plays. Additionally, the Author articulates the epistemological rationale for pre-publication peer-review and the inherent limitations of the system as a scientific quality-control mechanism. (...) The Article explores recent changes in science, in scientific publishing, and in the academy that have put the system under strain. The Author argues that Justice Blackmun's advice to courts - that peer-reviewed publication is relevant, but is not dispositive - is of little practical help. Instead, the Author suggests questions that courts should ask in assessing the significance of the fact that testimony is, or is not, based on peer-reviewed publication and illustrates with reference to another Bendectin case, Blum v. Merrell Dow Pharmaceuticals, Inc., where some of these questions were asked. (shrink)

Pre-publication certification through peer review stands in need of philosophical examination. In this paper, philosopher-psychologist Steven James Bartlett recalls the arguments marshalled four hundred years ago by English poet John Milton against restraint of publication by the "gatekeepers of publication," AKA today's peer reviewers.

Two important aspects of the relationship between peer review and innovation includes the acceptance of articles for publication in journals and the assessment of applications for grants for the funding of research work. While there are well-known examples of the rejection by journals of first choice of many papers that have radically changed the way we think about the world outside ourselves, such papers do get published eventually, however tortuous the process required. With grant applications the situation differs (...) in that the refusal of a grant necessarily curtails the possible research that may be attempted. Here there are many reasons for conservatism and reservation as to the ability of a grant allocation process based on peer review to deliver truly innovative investigations. Other methods are needed; although such methods need not be applied across the board, they should constitute the methods whereby some 10–20% of the grant monies are assigned. The nomination of prizes for specific accomplishments is one way of achieving innovation although this presumes that investigators or institution already have available the money necessary to effect the innovations; otherwise it is a question of the selection and funding of particular individuals or institutions and requiring them to solve particular problems that are set in the broadest of terms. (shrink)

Peer review is a widely accepted instrument for raising the quality of science. Peer review limits the enormous unstructured influx of information and the sheer amount of dubious data, which in its absence would plunge science into chaos. In particular, peer review offers the benefit of eliminating papers that suffer from poor craftsmanship or methodological shortcomings, especially in the experimental sciences. However, we believe that peer review is not always appropriate for the (...) evaluation of controversial hypothetical science. We argue that the process of peer review can be prone to bias towards ideas that affirm the prior convictions of reviewers and against innovation and radical new ideas. Innovative hypotheses are thus highly vulnerable to being “filtered out” or made to accord with conventional wisdom by the peer review process. Consequently, having introduced peer review, the Elsevier journal Medical Hypotheses may be unable to continue its tradition as a radical journal allowing discussion of improbable or unconventional ideas. Hence we conclude by asking the publisher to consider re-introducing the system of editorial review to Medical Hypotheses. (shrink)

Peer review has long been regarded as the gold standard of scientific publication, essential to the integrity of science itself. But, as any publishing scientist knows, peer review has its downside, including long delays and reviewer bias. Until the coming of the Internet, there appeared to be no alternative. Now, articles appear online as preprints almost immediately upon submission. But they lack peer review and thus their scientific standing can be questioned. Post-publication discussion platforms (...) such as PubPeer have proven useful, but are no substitute for pre-publication peer review. Nevertheless, some may be tempted to believe that peer review can now be done without. This article challenges that view by analyzing a recent, non-peer-reviewed article in Skeptical Enquirer, a magazine published by the Committee for Skeptical Enquiry (CSI). The article, “Sodom Meteor Strike Claims Should Be Taken With a Pillar of Salt,” casts doubt on one of the most widely read scientific articles of the last decade and provides a stern warning of the cost of abandoning peer review. (shrink)

At present, peer review is the most common method used by funding agencies to make decisions about resource allocation. But how reliable, efficient, and fair is it in practice? The ex ante evaluation of scientific novelty is a fundamentally uncertain endeavor; bias and chance are embedded in the final outcome. In the current study, I will examine some of the most central problems of peer review and highlight the possible benefits of using a lottery as an (...) alternative decision-making mechanism. Lotteries are driven by chance, not reason. The argument made in the study is that the epistemic landscape could benefit in several respects by using a lottery, thus avoiding all types of bias, disagreement, and other limitations associated with the peer review process. Funding agencies could form a pool of funding applicants who have minimal qualification levels and then select randomly within that pool. The benefits of a lottery would not only be that it saves time and resources, but also that it contributes to a more dynamic selection process and increases the epistemic diversity, fairness, and impartiality within academia. (shrink)

In a recent article in the British Journal for the Philosophy of Science, Heesen and Bright argue that prepublication peer review should be abolished and replaced with postpublication peer review (provided the matter is judged purely on epistemic grounds). In this article, I show that there are three problems with their argument. First, it fails to consider the epistemic cost of implementing the change to postpublication peer review. Second, it fails to consider some potential (...) epistemic benefits of prepublication peer review, which involve avoiding bias. Third, it fails to consider some potential epistemic disadvantages of postpublication peer review, which stem from the greater number of papers that would be published under that system. (shrink)

Peer review process helps in evaluating and validating of research that is published in the journals. U.S. Office of Research Integrity reported that data fraudulence was found to be involved in 94% cases of misconduct from 228 identified articles between 1994–2012. If fraud in published article are significantly as high as reported, the question arise in mind, were these articles peer reviewed? Another report said that the reviewers failed to detect 16 cases of fabricated article of Jan (...) Hendrick Schon. Superficial peer reviewing process does not reveals suspicion of misconduct. Lack of knowledge of systemic review process not only demolish the academic integrity in publication but also loss the trust of the people of the institution, the nation, and the world. The aim of this review article is to aware stakeholders specially novice reviewers about the peer review system. Beginners will understand how to review an article and they can justify better action choices in dealing with reviewing an article. (shrink)

Mission-oriented climate change research is often unverifiable. Therefore, many stakeholders look to peer-reviewed climate change research for trustworthy information about deeply uncertain and impactful phenomena. This is because peer-review signals that research has been vetted for scientific standards like reproducibility and replicability. Here we evaluate the transparency of research methodologies in mission-oriented computational climate research. We find that only five percent of our sample meets the minimal standard of fully open data and code required for reproducibility and (...) replicability. The widespread lack of minimal openness standards in peer-reviewed climate change research jeopardizes scientific credibility and risks poor societal outcomes as we manage growing climate risks. The scientific community can and must do better. We, the authors, outline necessary openness standards that we commit to as researchers, reviewers, and readers. We invite you to join us. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the (...) normative implications of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

Background: Editorial peer review is universally used but little studied. We examined the relationship between external reviewers' recommendations and the editorial outcome of manuscripts undergoing external peer-review at the Journal of General Internal Medicine. Methodology/Principal Findings: We examined reviewer recommendations and editors' decisions at JGIM between 2004 and 2008. For manuscripts undergoing peer review, we calculated chance-corrected agreement among reviewers on recommendations to reject versus accept or revise. Using mixed effects logistic regression models, we (...) estimated intra-class correlation coefficients at the reviewer and manuscript level. Finally, we examined the probability of rejection in relation to reviewer agreement and disagreement. The 2264 manuscripts sent for external review during the study period received 5881 reviews provided by 2916 reviewers; 28% of reviews recommended rejection. Chance corrected agreement on rejection among reviewers was 0.11. In mixed effects models adjusting for study year and manuscript type, the reviewer-level ICC was 0.23 and the manuscript-level ICC was 0.17. The editors' overall rejection rate was 48%: 88% when all reviewers for a manuscript agreed on rejection and 20% when all reviewers agreed that the manuscript should not be rejected. Conclusions/Significance: Reviewers at JGIM agreed on recommendations to reject vs. accept/revise at levels barely beyond chance, yet editors placed considerable weight on reviewers' recommendations. Efforts are needed to improve the reliability of the peer-review process while helping editors understand the limitations of reviewers' recommendations. © 2010 Kravitz et al. (shrink)

A relatively high incidence of unsatisfactory review decisions is widely recognised and acknowledged as ‘the peer review problem’. Factors contributing to this problem are identified and examined. Specific examples of unreasonable rejection are considered. It is concluded that weaknesses of the ‘peer review’ system are significant and that they are well known or readily recognisable but that necessary counter-measures are not always enforced. Careful management is necessary to discount hollow opinion or error in review (...) comment. Review and referee functions should be quite separate. (shrink)

That there is a “crisis of peer review” at the moment is not in dispute, but sufficient attention has not yet been paid to the normative potential that lies in current calls for reform. In contrast to approaches to “fixing” the problems in peer review, which tend to maintain the status quo in terms of professionalising opportunities, this paper addresses the needs of philosophers and how peer‐review reform can be an opportunity to improve the (...) academic discipline of philosophy, whereby progress is understood as making the discipline more fair to the global academic community and more conducive to the flourishing of academic philosophers. The paper evaluates recent categories of relevant norms and correlating reforms. In conclusion, it recommends that philosophy pursue the norms of transparency and democracy explicitly when proposing peer‐review reform and suggest that proposals for forum‐based models of peer review are most likely to support those norms. (shrink)

Recent science funding policy scholars and practitioners have advocated for the use of lotteries, or elements of random chance, as supplementations of traditional peer review for evaluating grant applications. One of the primary motivations for lotteries is their purported openness to innovative research. The purpose of this paper is to argue that current proponents of funding science by lottery overestimate the viability of peer review and thus unduly restrict the scope of lotteries in science funding practice. (...) I further show how this analysis suggests a different way of introducing lotteries into science funding policy. (shrink)

Retractions of COVID-19 literature in both preprints and the peer-reviewed literature serve as a reminder that there are still challenging issues underlying the integrity of the biomedical literature. The risks to academia become larger when such retractions take place in high-ranking biomedical journals. In some cases, retractions result from unreliable or nonexistent data, an issue that could easily be avoided by having open data policies, but there have also been retractions due to oversight in peer review and (...) editorial verification. As COVID-19 continues to affect academics and societies around the world, failures in peer review might also constitute a public health risk. The effectiveness by which COVID-19 literature is corrected, including through retractions, depends on the stringency of measures in place to detect errors and to correct erroneous literature. It also relies on the stringent implementation of open data policies. (shrink)

Previous research has found that funding disparities are driven by applications’ final impact scores and that only a portion of the black/white funding gap can be explained by bibliometrics and topic choice. Using National Institutes of Health R01 applications for council years 2014–2016, we examine assigned reviewers’ preliminary overall impact and criterion scores to evaluate whether racial disparities in impact scores can be explained by application and applicant characteristics. We hypothesize that differences in commensuration—the process of combining criterion scores into (...) overall impact scores—disadvantage black applicants. Using multilevel models and matching on key variables including career stage, gender, and area of science, we find little evidence for racial disparities emerging in the process of combining preliminary criterion scores into preliminary overall impact scores. Instead, preliminary criterion scores fully account for racial disparities—yet do not explain all of the variability—in preliminary overall impact scores. (shrink)

This Open Access book about the Zadeh Project demonstrates and explores a core question in clinical ethics: how can ethics consultants be accountable in the face of a robust plurality of ethical standpoints, especially those that underwrite practices and methods for doing ethics consultation as well as those viewpoints and values encountered in daily clinical ethics practice? Underscoring this question is the recognition that the field of clinical ethics consultation has arrived at a crucial point in its maturation. Many efforts (...) are underway to more formally “professionalize” the field, with most aimed toward stabilizing a specific set of institutional considerations. Stretched between these institutional and practical initiatives resides a crucial set of of ethical considerations, chief among them the meaning and scope of responsibility for clinical ethics consultants. Developed around a long-form case scenario, the Zadeh Project provides a multi-layered series of “peer-reviews”: critique of the actions of the case scenario’s ethics consultant; reflection on clinical ethics method; examination of the many ways that commitments to method and practice can, and do, intersect, overlap, and alter one another. The design and format of this book thus models a key element for clinical ethics practice: the need and ability to provide careful and thoughtful explanation of core moral considerations that emerge among diverse standpoints. Specifically designed for those studying to become and those who are ethics consultants, this book, with its innovative and multi-layered approach, allows readers to share a peer-review-like experience that shows accountability to be what it is, an ethical, not merely procedural or administrative, undertaking. (shrink)

Peer review is an opportunity to perform an unlawful censorship which ensures that no apostate notion ever get published in mainstream journals. Or such peer review censorship is an opportunity to steal any content and to claim afterward the priority of first publication. And last but not least, the peer review is an academic tool to promote the mainstream pseudoscience.

An error in the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, which wrongly predicted the disappearance of Himalayan glaciers by 2035, fueled doubts about the authority, honesty and rigor of the IPCC as a leading institution in climate science and, correspondingly, raised questions about whether global warming is anything more than a hoax put forward by environmentalists. The late and confusing reaction of the IPCC to these allegations only worsened the matter. By comparing assessment reports issued by (...) the IPCC, this paper asks the question: ‘Why, despite extensive peer review, did the Himalayan glacier melting rate error get published?’ I chronicle exactly what happened, consider why it happened as it did, and show how these answers require us to rethink the concept of ‘peer review’ in scientific practice. I also identify several future directions for peer review if it wants to stay ahead of the game, bearing in mind the forthcoming IPCC Assessment Reports. (shrink)