The replication crisis has prompted many to call for statistical reform within the psychological sciences. Here we examine issues within Frequentist statistics that may have led to the replication crisis, and we examine the alternative—Bayesian statistics—that many have suggested as a replacement. The Frequentist approach and the Bayesian approach offer radically different perspectives on evidence and inference with the Frequentist approach prioritising error control and the Bayesian approach offering a formal method for quantifying the relative strength of evidence (...) for hypotheses. We suggest that rather than mere statistical reform, what is needed is a better understanding of the different modes of statistical inference and a better understanding of how statistical inference relates to scientific inference. (shrink)

We provide statistical inference for measures of predictive success. These measures are frequently used to evaluate and compare the performance of different models of individual and group decision making in experimental and revealed preference studies. We provide a brief illustration of our findings by comparing the predictive success of different revealed preference tests for models of intertemporal decision making. This demonstrates that it is possible to compare the predictive success of different models in a statistically meaningful way.

We analyze the quantum mechanical measuring process from the standpoint of information theory. Statistical inference is used in order to define the most likely state of the measured system that is compatible with the readings of the measuring instrument and the a priori information about the correlations between the system and the instrument. This approach has the advantage that no reference to the time evolution of the combined system need be made. It must, however, be emphasized that the (...) result is to be interpreted as the statistically inferred state of the original system rather than the state of the system after measurement. The phenomenon of “reduction of states” appears in this light as a consequence of incomplete information rather than the physical interaction between measured system and measuring instrument. (shrink)

Statistical inference is often justified by long-run properties of the sampling distributions, such as the repeated sampling rationale. These are frequentist justifications of statistical inference. I argue, in line with existing philosophical literature, but against a widespread image in empirical science, that these justifications are flawed. Then I propose a novel interpretation of probability in statistics, the artefactual interpretation. I believe that this interpretation is able to bridge the gap between statistical probability calculations and rational (...) decisions on the basis of observed data. The artefactual interpretation is able to justify statistical inference without making any assumptions about probability in the material world. (shrink)

In order to account for how children can generalize words beyond a very limited set of labeled examples, Bloom's proposal of word learning requires two extensions: a better understanding of the “general learning and memory abilities” involved, and a principled framework for integrating multiple conflicting constraints on word meaning. We propose a framework based on Bayesian statistical inference that meets both of those needs.

Much research on cognitive development focuses either on early-emerging domain-specific knowledge or domain-general learning mechanisms. However, little research examines how these sources of knowledge interact. Previous research suggests that young infants can make inferences from samples to populations (Xu & Garcia, 2008) and 11- to 12.5-month-old infants can integrate psychological and physical knowledge in probabilistic reasoning (Teglas, Girotto, Gonzalez, & Bonatti, 2007; Xu & Denison, 2009). Here, we ask whether infants can integrate a physical constraint of immobility into a (...) class='Hi'>statistical inference mechanism. Results from three experiments suggest that, first, infants were able to use domain-specific knowledge to override statistical information, reasoning that sometimes a physical constraint is more informative than probabilistic information. Second, we provide the first evidence that infants are capable of applying domain-specific knowledge in probabilistic reasoning by using a physical constraint to exclude one set of objects while computing probabilities over the remaining sets. (shrink)

One of Ian Hacking's earliest publications, this book showcases his early ideas on the central concepts and questions surrounding statistical reasoning. He explores the basic principles of statistical reasoning and tests them, both at a philosophical level and in terms of their practical consequences for statisticians. Presented in a fresh twenty-first-century series livery, and including a specially commissioned preface written by Jan-Willem Romeijn, illuminating its enduring importance and relevance to philosophical enquiry, Hacking's influential and original work has been (...) revived for a new generation of readers. (shrink)

Frequentist inference typically is described in terms of hypothetical repeated sampling but there are advantages to an interpretation that uses a single random sample. Contemporary examples are given that indicate probabilities for random phenomena are interpreted as classical probabilities, and this interpretation of equally likely chance outcomes is applied to statistical inference using urn models. These are used to address Bayesian criticisms of frequentist methods. Recent descriptions of p-values, confidence intervals, and power are viewed through the lens (...) of classical probability based on a single random sample from the population. (shrink)

The major competing statistical paradigms share a common remarkable but unremarked thread: in many of their inferential applications, different probability interpretations are combined. How this plays out in different theories of inference depends on the type of question asked. We distinguish four question types: confirmation, evidence, decision, and prediction. We show that Bayesian confirmation theory mixes what are intuitively “subjective” and “objective” interpretations of probability, whereas the likelihood-based account of evidence melds three conceptions of what constitutes an “objective” (...) probability. (shrink)

Although written in Japanese, what the statistical inference is philosophically investigated.

Scientists and Bayesian statisticians often study hypotheses that they know to be false. This creates an interpretive problem because the Bayesian probability of a hypothesis is supposed to represent the probability that the hypothesis is true. I investigate whether Bayesianism can accommodate the idea that false hypotheses are sometimes approximately true or that some hypotheses or models can be closer to the truth than others. I argue that the idea that some hypotheses are approximately true in an absolute sense is (...) hard to square with Bayesianism, but that the notion that some hypotheses are comparatively closer to the truth than others can be made compatible with Bayesianism, and that this provides an adequate and potentially useful solution to the interpretive problem. Finally, I compare my ``verisimilitude'' solution to the interpretive problem with a ``counterfactual'' solution recently proposed by Jan Sprenger. (shrink)

NHSTP is embedded in the research of “cognitive science.” Its use is based on unstated assumptions about the practices of sampling, “operationalizing,” and using group data. NHSTP has facilitated both research and theorizing – research findings of limited interest – diverse theories that seldom complement one another. Alternative methods are available for data acquisition and analysis, and for assessing the “truth- value” of generalizations.

Abstract Based on recalling two characteristic features of Bayesian statistical inference in commutative probability theory, a stability property of the inference is pointed out, and it is argued that that stability of the Bayesian statistical inference is an essential property which must be preserved under generalization of Bayesian inference to the non?commutative case. Mathematical no?go theorems are recalled then which show that, in general, the stability can not be preserved in non?commutative context. Two possible (...) interpretations of the impossibility of generalization of Bayesian statistical inference to the non?commutative case are offered, none of which seems to be completely satisfying. (shrink)

In May of 1973 we organized an international research colloquium on foundations of probability, statistics, and statistical theories of science at the University of Western Ontario. During the past four decades there have been striking formal advances in our understanding of logic, semantics and algebraic structure in probabilistic and statistical theories. These advances, which include the development of the relations between semantics and metamathematics, between logics and algebras and the algebraic-geometrical foundations of statistical theories (especially in the (...) sciences), have led to striking new insights into the formal and conceptual structure of probability and statistical theory and their scientific applications in the form of scientific theory. The foundations of statistics are in a state of profound conflict. Fisher's objections to some aspects of Neyman-Pearson statistics have long been well known. More recently the emergence of Baysian statistics as a radical alternative to standard views has made the conflict especially acute. In recent years the response of many practising statisticians to the conflict has been an eclectic approach to statistical inference. Many good statisticians have developed a kind of wisdom which enables them to know which problems are most appropriately handled by each of the methods available. The search for principles which would explain why each of the methods works where it does and fails where it does offers a fruitful approach to the controversy over foundations. (shrink)

The main thesis of the paper is that in the case of modern statistics, the differences between the various concepts of models were the key to its formative controversies. The mathematical theory of statistical inference was mainly developed by Ronald A. Fisher, Jerzy Neyman, and Egon S. Pearson. Fisher on the one side and Neyman–Pearson on the other were involved often in a polemic controversy. The common view is that Neyman and Pearson made Fisher's account more stringent mathematically. (...) It is argued, however, that there is a profound theoretical basis for the controversy: both sides held conflicting views about the role of mathematical modelling. At the end, the influential programme of Exploratory Data Analysis is considered to be advocating another, more instrumental conception of models. Introduction Models in statistics—‘of what population is this a random sample?’ The fundamental lemma Controversy about models Exploratory data analysis as a model-critical approach. (shrink)

An accessible and illuminating exploration of the conceptual basisof scientific and statistical inference and the practical impact this has on conducting psychological research. The book encourages a critical discussion of the different approaches and looks at some of the most important thinkers and their influence.

This paper addresses the problem that Bayesian statistical inference cannot accommodate theory change, and proposes a framework for dealing with such changes. It first presents a scheme for generating predictions from observations by means of hypotheses. An example shows how the hypotheses represent the theoretical structure underlying the scheme. This is followed by an example of a change of hypotheses. The paper then presents a general framework for hypotheses change, and proposes the minimization of the distance between hypotheses (...) as a rationality criterion. Finally the paper discusses the import of this for Bayesian statistical inference. (shrink)

Probability and inverse inference; Neyman-Pearson theory; Fisherian significance testing; The fiducial argument: one parameter; The fiducial argument: several parameters; Ian hacking's theory; Henry Kyburg's theory; Relevance and experimental design.

The main thesis of the paper is that in the case of modern statistics, the differences between the various concepts of models were the key to its formative controversies. The mathematical theory of statistical inference was mainly developed by Ronald A. Fisher, Jerzy Neyman, and Egon S. Pearson. Fisher on the one side and Neyman–Pearson on the other were involved often in a polemic controversy. The common view is that Neyman and Pearson made Fisher's account more stringent mathematically. (...) It is argued, however, that there is a profound theoretical basis for the controversy: both sides held conflicting views about the role of mathematical modelling. At the end, the influential programme of Exploratory Data Analysis is considered to be advocating another, more instrumental conception of models. 1. Introduction2. Models in statistics—‘of what population is this a random sample?’3. The fundamental lemma4. Controversy about models5. Exploratory data analysis as a model-critical approach. (shrink)

Historical records show that there was no real concept of probability in Europe before the mid-seventeenth century, although the use of dice and other randomizing objects was commonplace. Ian Hacking presents a philosophical critique of early ideas about probability, induction, and statistical inference and the growth of this new family of ideas in the fifteenth, sixteenth, and seventeenth centuries. Hacking invokes a wide intellectual framework involving the growth of science, economics, and the theology of the period. He argues (...) that the transformations that made it possible for probability concepts to emerge have constrained all subsequent development of probability theory and determine the space within which philosophical debate on the subject is still conducted. First published in 1975, this edition includes an introduction that contextualizes his book in light of developing philosophical trends. Ian Hacking is the winner of the Holberg International Memorial Prize 2009. (shrink)

Ian Hacking here presents a philosophical critique of early ideas about probability, induction and statistical inference and the growth of this new family of ...

Can the axioms of probability theory and the classical patterns of statistical inference ever be falsified by observation? Various possible answers to this question are examined in a set theoretical context and in relation to the findings of microphysics.

Surprise—operationalized as looking time—has a long history in developmental research, providing a window into the perception and cognition of infants. Recently, however, a number of developmental researchers have considered infants’ and children's surprise in its own right. This article reviews empirical evidence and computational models of complex statistical inferences underlying surprise, and discusses how these findings relate to the role that surprise appears to play as a catalyst for learning.

I make two points in this commentary on Carey (2009). First, it may be too soon to conclude that core cognition is innate. Recent advances in computational cognitive science and developmental psychology suggest possible mechanisms for developing inductive biases. Second, there is another possible answer to Fodor's challenge – if concepts are merely mental tokens, then cognitive scientists should spend their time on developing a theory of belief fixation instead.

The three main approaches in statistical inference—classical statistics, Bayesian and likelihood—are in current use in phylogeny research. The three approaches are discussed and compared, with particular emphasis on theoretical properties illustrated by simple thought-experiments. The methods are problematic on axiomatic grounds, extra-mathematical grounds relating to the use of a prior or practical grounds. This essay aims to increase understanding of these limits among those with an interest in phylogeny.

The present work is an expansion of the author's doctoral dissertation, entitled "The Fiducial Argument," completed at Columbia University in 1975. Seidenfeld's principal objective, as stated in the Preface, is to "reconstruct and evaluate Fisherian statistics, with special attention to Fisher's idea of fiducial probability as it pertains to inverse inference.".

This paper analyzes Deborah Mayo's error-statistical (ES) account of scientific evidence in order to clarify the kinds of "material postulates" it requires and to explain how those assumptions function. A secondary aim is to explain and illustrate the importance of the security of an inference. After finding that, on the most straightforward reading of the ES account, it does not succeed in its stated aims, two remedies are considered: either relativize evidence claims or introduce stronger assumptions. The choice (...) between these approaches turns on the value attached to two aims of inquiry that are in tension: drawing strong, informative conclusions and reasoning securely. (shrink)

Historical records show that there was no real concept of probability in Europe before the mid-seventeenth century, although the use of dice and other randomizing objects was commonplace. Ian Hacking presents a philosophical critique of early ideas about probability, induction, and statistical inference and the growth of this new family of ideas in the fifteenth, sixteenth, and seventeenth centuries. Hacking invokes a wide intellectual framework involving the growth of science, economics, and the theology of the period. He argues (...) that the transformations that made it possible for probability concepts to emerge have constrained all subsequent development of probability theory and determine the space within which philosophical debate on the subject is still conducted. First published in 1975, this edition includes an introduction that contextualizes his book in light of developing philosophical trends. Ian Hacking is the winner of the Holberg International Memorial Prize 2009. (shrink)

The use of null hypothesis significance testing (NHST) in psychology has been under sustained attack, despite its reliable use in the notably successful, so-called "hard" areas of psychology, such as perception and cognition. I argue that, in contrast to merely methodological analyses of hypothesis testing (in terms of "test severity," or other confirmation-theoretic notions), only a patently metaphysical position can adequately capture the uneven but undeniable successes of theories in "hard psychology." I contend that Measured Realism satisfies this description, and (...) characterizes the role of NHST in hard psychology. (shrink)