Understanding the Replication Crisis as a Base Rate Fallacy

British Journal for the Philosophy of Science 72 (4):965-993 (2021)
  Copy   BIBTEX

Abstract

The replication (replicability, reproducibility) crisis in social psychology and clinical medicine arises from the fact that many apparently well-confirmed experimental results are subsequently overturned by studies that aim to replicate the original study. The culprit is widely held to be poor science: questionable research practices, failure to publish negative results, bad incentives, and even fraud. In this article I argue that the high rate of failed replications is consistent with high-quality science. We would expect this outcome if the field of science in question produces a high proportion of false hypotheses prior to testing. If most of the hypotheses under test are false, then there will be many false hypotheses that are apparently supported by the outcomes of well conducted experiments and null hypothesis significance tests with a type-I error rate (α) of 5%. Failure to recognize this is to commit the fallacy of ignoring the base rate. I argue that this is a plausible diagnosis of the replication crisis and examine what lessons we thereby learn for the future conduct of science.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 91,386

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Is the replication crisis a base-rate fallacy?Bengt Autzen - 2021 - Theoretical Medicine and Bioethics 42 (5):233-243.
Understanding the replication crisis as a base rate fallacy.Alexander Bird - 2018 - British Journal for the Philosophy of Science:000-000.
Kuhnian theory-choice and virtue convergence: Facing the base rate fallacy.Samuel Schindler - 2017 - Studies in History and Philosophy of Science Part A 64:30-37.
Exhuming the No-Miracles Argument.Colin Howson - 2013 - Analysis 73 (2):205-211.
No Surprises.Ian Wells - 2019 - Erkenntnis 86 (2):389-406.
Does the miracle argument embody a base rate fallacy?Cornelis Menke - 2014 - Studies in History and Philosophy of Science Part A 45:103-108.
Pragmatically before ecologically valid tasks.Laura Macchi - 1997 - Behavioral and Brain Sciences 20 (4):778-779.
Realist Ennui and the Base Rate Fallacy.P. D. Magnus & Craig Callender - 2004 - Philosophy of Science 71 (3):320-338.
A Replication Crisis in Mathematics?Anthony Bordg - 2021 - The Mathematical Intelligencer 1.

Analytics

Added to PP
2022-04-07

Downloads
88 (#189,215)

6 months
26 (#109,390)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Alexander James Bird
Cambridge University

Citations of this work

The new demarcation problem.Bennett Holman & Torsten Wilholt - 2022 - Studies in History and Philosophy of Science Part A 91 (C):211-220.
Science Communication and the Problematic Impact of Descriptive Norms.Uwe Peters - 2023 - British Journal for the Philosophy of Science 74 (3):713-738.
The role of replication in psychological science.Samuel C. Fletcher - 2021 - European Journal for Philosophy of Science 11 (1):1-19.
The New Worries about Science.Janet A. Kourany - 2022 - Canadian Journal of Philosophy 52 (3):227-245.

View all 12 citations / Add more citations

References found in this work

The Logic of Scientific Discovery.Karl Popper - 1959 - Studia Logica 9:262-265.
Can the Behavioral Sciences Self-correct? A Social Epistemic Study.Felipe Romero - 2016 - Studies in History and Philosophy of Science Part A 60 (C):55-69.

View all 10 references / Add more references