After some general remarks about the interrelation between philosophical and statistical thinking, the discussion centres largely on significance tests. These are defined as the calculation of p-values rather than as formal procedures for ‘acceptance‘ and ‘rejection‘. A number of types of null hypothesis are described and a principle for evidential interpretation set out governing the implications of p- values in the specific circumstances of each application, as contrasted with a long-run interpretation. A number of more complicated situ- ations are discussed (...) in which modification of the simple p-value may be essential. (shrink)

One approach to validating simulation models is to formally compare model outputs with independent data. We consider such model validation from the point of view of Frequentist statistics. A range of estimates and tests of goodness of fit have been advanced. We review these approaches, and demonstrate that some of the tests suffer from difficulties in interpretation because they rely on the null hypothesisHypothesis that the model is similar to the observationsObservations. This reliance creates two unpleasant possibilities, namely, (...) a model could be spuriously validated when dataData are too few, or inappropriately rejected when data are too many. Finally, these tests do not allow a principled declaration of what a reasonable level of difference would be considering the purposes to which the model will be put. We consider equivalence tests, and demonstrate that they do not suffer from the previously identified shortcomings. We provide two case studies to illustrate the claims of the chapter. (shrink)

Frequentist inference typically is described in terms of hypothetical repeated sampling but there are advantages to an interpretation that uses a single random sample. Contemporary examples are given that indicate probabilities for random phenomena are interpreted as classical probabilities, and this interpretation of equally likely chance outcomes is applied to statistical inference using urn models. These are used to address Bayesian criticisms of frequentist methods. Recent descriptions of p-values, confidence intervals, and power are viewed through the lens of (...) classical probability based on a single random sample from the population. (shrink)

I investigate the extent to which perspectival realism agrees with frequentist statistical methodology and philosophy, with an emphasis on J. Neyman’s views. Based on the example of the stopping rule problem, I show how PR can naturally be associated with frequentist statistics in general. I also show that there are some aspects of Neyman’s thought that seem to confirm PR and others that disconfirm it. I argue that epistemic PR is consistent with Neyman’s frequentism to a satisfactory (...) degree and that on the grounds of Neyman’s frequentist methodology one is dealing with genuine and non-trivial perspectives. I also argue that universality and normativity of PR are weakened when analysed from the point of view of Neyman’s statistical methodology. (shrink)

Amidst long-running debates within the field, high energy physics has adopted a statistical methodology that primarily employs standard frequentist techniques such as significance testing and confidence interval estimation, but incorporates Bayesian methods for limited purposes. The discovery of the Higgs boson has drawn increased attention to the statistical methods employed within HEP. Here I argue that the warrant for the practice in HEP of relying primarily on frequentist methods can best be understood as pragmatic, in the sense that (...) statistical methods are chosen for their suitability for the practical demands of experimental HEP, rather than reflecting a commitment to a particular epistemic framework. In particular, I argue that understanding the statistical methodology of HEP through the perspective of pragmatism clarifies the role of and rationale for significance testing in the search for new phenomena such as the Higgs boson. (shrink)

The debates between Bayesian, frequentist, and other methodologies of statistics have tended to focus on conceptual justifications, sociological arguments, or mathematical proofs of their long run properties. Both Bayesian statistics and frequentist (“classical”) statistics have strong cases on these grounds. In this article, we instead approach the debates in the “Statistics Wars” from a largely unexplored angle: simulations of different methodologies’ performance in the short to medium run. We conducted a large number of simulations (...) using a straightforward decision problem based around tossing a coin with unknown bias and then placing bets. In this simulation, we programmed four players, inspired by Bayesian statistics, frequentist statistics, Jon Williamson’s version of Objective Bayesianism, and a player who simply extrapolates from observed frequencies to general frequencies. The last player functions as a benchmark: a statistical methodology should at least outperform a crude form of induction. We focused on the performance of these methodologies in guiding the players towards good decisions. Unlike an earlier simulation study of this type, we found no systematic difference in performance between the Bayesian and frequentist players, provided the Bayesian used a flat prior and the frequentist used a low confidence level. Unlike that study, we were able to use Big Data methods to mitigate problems of random error in the simulation results. The Williamsonian player, who is a novel element of our study, also had no systematic differences in their performance, provided that they use a low confidence level. These players performed similarly even in the very short run, when players were making different decisions. Our study indicates that all three methodologies should be taken seriously by philosophers and practitioners of statistics. However, the frequentist and Williamsonian performed poorly when their confidence levels were high, and the Bayesian was surprisingly harmed by biased priors, providing some unexpected lessons for these methodologies when facing this type of decision problem. (shrink)

Statistical inference is often justified by long-run properties of the sampling distributions, such as the repeated sampling rationale. These are frequentist justifications of statistical inference. I argue, in line with existing philosophical literature, but against a widespread image in empirical science, that these justifications are flawed. Then I propose a novel interpretation of probability in statistics, the artefactual interpretation. I believe that this interpretation is able to bridge the gap between statistical probability calculations and rational decisions on the (...) basis of observed data. The artefactual interpretation is able to justify statistical inference without making any assumptions about probability in the material world. (shrink)

This book analyzes the origins of statistical thinking as well as its related philosophical questions, such as causality, determinism or chance. Bayesian and frequentist approaches are subjected to a historical, cognitive and epistemological analysis, making it possible to not only compare the two competing theories, but to also find a potential solution. The work pursues a naturalistic approach, proceeding from the existence of numerosity in natural environments to the existence of contemporary formulas and methodologies to heuristic pragmatism, a concept (...) introduced in the book's final section. This monograph will be of interest to philosophers and historians of science and students in related fields. Despite the mathematical nature of the topic, no statistical background is required, making the book a valuable read for anyone interested in the history of statistics and human cognition. (shrink)

Philosophers now seem to agree that frequentism is an untenable strategy to explain the meaning of probabilities. Nevertheless, I want to revive frequentism, and I will do so by grounding probabilities on typicality in the same way as the thermodynamic arrow of time can be grounded on typicality within statistical mechanics. This account, which I will call typicality frequentism, will evade the major criticisms raised against previous forms of frequentism. In this theory, probabilities arise within a physical theory from statistical (...) behavior of almost all initial conditions. The main advantage of typicality frequentism is that it shows which kinds of probabilities (that also have empirical relevance) can be derived from physics. Although one cannot recover all probability talk in this account, this is rather a virtue than a vice, because it shows which types of probabilities can in fact arise from physics and which types need to be explained in different ways, thereby opening the path for a pluralistic account of probabilities. (shrink)

I show that frequentism, as an explanation of probability in classical statistical mechanics, can be extended in a natural way to a decoherent quantum history space, the analogue of a classical phase space. The result is a form of finite frequentism, in which Gibbs’ concept of an infinite ensemble of gases is replaced by the quantum state expressed as a superposition of a finite number of decohering microstates. It is a form of finite and actual frequentism (as opposed to hypothetical (...) frequentism) insofar as all the microstates exist, in keeping with the decoherence-based Everett interpretation, and some versions of pilot-wave theory. (shrink)

The orthodox view in statistics has it that frequentism and Bayesianism are diametrically opposed—two totally incompatible takes on the problem of statistical inference. This paper argues to the contrary that the two approaches are complementary and need to mesh if probabilistic reasoning is to be carried out correctly.

The main objective of the paper is to propose a frequentist interpretation of probability in the context of model-based induction, anchored on the Strong Law of Large Numbers (SLLN) and justifiable on empirical grounds. It is argued that the prevailing views in philosophy of science concerning induction and the frequentist interpretation of probability are unduly influenced by enumerative induction, and the von Mises rendering, both of which are at odds with frequentist model-based induction that dominates current practice. (...) The differences between the two perspectives are brought out with a view to defend the model-based frequentist interpretation of probability against certain well-known charges, including [i] the circularity of its definition, [ii] its inability to assign ‘single event’ probabilities, and [iii] its reliance on ‘random samples’. It is argued that charges [i]–[ii] stem from misidentifying the frequentist ‘long-run’ with the von Mises collective. In contrast, the defining characteristic of the long-run metaphor associated with model-based induction is neither its temporal nor its physical dimension, but its repeatability (in principle); an attribute that renders it operational in practice. It is also argued that the notion of a statistical model can easily accommodate non-IID samples, rendering charge [iii] simply misinformed. (shrink)

The replication crisis has prompted many to call for statistical reform within the psychological sciences. Here we examine issues within Frequentist statistics that may have led to the replication crisis, and we examine the alternative—Bayesian statistics—that many have suggested as a replacement. The Frequentist approach and the Bayesian approach offer radically different perspectives on evidence and inference with the Frequentist approach prioritising error control and the Bayesian approach offering a formal method for quantifying the relative (...) strength of evidence for hypotheses. We suggest that rather than mere statistical reform, what is needed is a better understanding of the different modes of statistical inference and a better understanding of how statistical inference relates to scientific inference. (shrink)

In this paper I provide a frequentist philosophical-methodological solution for the stopping rule problem presented by Lindley & Phillips in 1976, which is settled in the ecological realm of testing koalas’ sex ratio. I deliver criteria for discerning a stopping rule, an evidence and a model that are epistemically more appropriate for testing the hypothesis of the case studied, by appealing to physical notion of probability and by analyzing the content of possible formulations of evidence, assumptions of models and (...) meaning of the ecological hypothesis. First, I show the difference in the evidence taken into account in different frequentist sampling procedures presented in the problem. Next, I discuss the inapplicability of the Carnapian principle of total evidence in deciding which formulation of evidence associated with a given sampling procedure and statistical model is epistemically more appropriate for testing the hypothesis in question. Then I propose a double-perspective (evidence and model) frequentist solution based on the choice of evidence which better corresponds to the investigated ecological hypothesis, as well as on the choice of a model that embraces less unrealistic ontological assumptions. Finally, I discuss two perspectives of the stopping rule dependence. (shrink)

This article calls into question the charge that frequentist testing is susceptible to the base-rate fallacy. It is argued that the apparent similarity between examples like the Harvard Medical School test and frequentist testing is highly misleading. A closer scrutiny reveals that such examples have none of the basic features of a proper frequentist test, such as legitimate data, hypotheses, test statistics, and sampling distributions. Indeed, the relevant error probabilities are replaced with the false positive/negative rates (...) that constitute deductive calculations based on known probabilities among events. As a result, the ampliative dimension of frequentist induction—learning from data about the underlying data-generating mechanism—is missing. *Received August 2009; revised January 2010. †To contact the author, please write to: Department of Economics, Virginia Tech, Blacksburg, VA 24061; e-mail: [email protected]. (shrink)

We analyse the issue of using prior information in frequentist statistical inference. For that purpose, we scrutinise different kinds of sampling designs in Jerzy Neyman’s theory to reveal a variety of ways to explicitly and objectively engage with prior information. Further, we turn to the debate on sampling paradigms (design-based vs. model-based approaches) to argue that Neyman’s theory supports an argument for the intermediate approach in the frequentism vs. Bayesianism debate. We also demonstrate that Neyman’s theory, by allowing non-epistemic (...) values to influence evidence collection and formulation of statistical conclusions, does not compromise the epistemic reliability of the procedures and may improve it. This undermines the value-free ideal of scientific inference. (shrink)

This paper deals with meta-statistical questions concerning frequentist statistics. In Sections 2 to 4 I analyse the dispute between Fisher and Neyman on the so called logic of statistical inference, a polemic that has been concomitant of the development of mathematical statistics. My conclusion is that, whenever mathematical statistics makes it possible to draw inferences, it only uses deductive reasoning. Therefore I reject Fisher's inductive approach to the statistical estimation theory and adhere to Neyman's deductive one. (...) On the other hand, I assert that Neyman-Pearson's testing theory, as well as Fisher's tests of significance, properly belong to decision theory, not to logic, neither deductive nor inductive. I then also disagree with Costantini's view of Fisher's testing model as a theory of hypothetico-deductive inferences.In Section 5 I disapprove Hacking1's evidentialists criticisms of the Neyman-Pearson's theory of statistics (NPT), as well as Hacking2's interpretation of NPT as a theory of probable inference. In both cases Hacking misses the point. I conclude, by claiming that Mayo's conception of the Neyman-Pearson's testing theory, as a model of learning from experience, does not purport any advantages over Neyman's behavioristic model. (shrink)

The origins of testing scientific models with statistical techniques go back to 18th century mathematics. However, the modern theory of statistical testing was primarily developed through the work of Sir R.A. Fisher, Jerzy Neyman, and Egon Pearson in the inter-war period. Some of Fisher's papers on testing were published in economics journals (Fisher, 1923, 1935) and exerted a notable influence on the discipline. The development of econometrics and the rise of quantitative economic models in the mid-20th century made statistical significance (...) testing a commonplace, albeit controversial tool within economics. -/- In the debate about significance testing, methodological controversies intertwine with epistemological issues and sociological developments. Our aim in this chapter is to expound these connections and to show how the use of, and the debate about, significance testing in economics differs from other social sciences, such as psychology. (shrink)

In statistics, there are two main paradigms: classical and Bayesian statistics. The purpose of this article is to investigate the extent to which classicists and Bayesians can agree. My conclusion is that, in certain situations, they cannot. The upshot is that, if we assume that the classicist is not allowed to have a higher degree of belief in a null hypothesis after he has rejected it than before, then he has to either have trivial or incoherent credences to (...) begin with or fail to update his credences by conditionalization. (shrink)

The paper develops models of statistical experiments that combine propensities with frequencies, the underlying theory being the branching space-times (BST) of Belnap (1992). The models are then applied to analyze Bell's theorem. We prove the so-called Bell-CH inequality via the assumptions of a BST version of Outcome Independence and of (non-probabilistic) No Conspiracy. Notably, neither the condition of probabilistic No Conspiracy nor the condition of Parameter Independence is needed in the proof. As the Bell-CH inequality is most likely experimentally falsified, (...) the choice is this: contrary to the appearances, experimenters cannot choose some measurement settings, or some transitions, with spacelike related initial events, are correlated; or both. (shrink)

Statistical tests of the primality of some numbers look similar to statistical tests of many nonmathematical, clearly empirical propositions. Yet interpretations of probability prima facie appear to preclude the possibility of statistical tests of mathematical propositions. For example, it is hard to understand how the statement that n is prime could have a frequentist probability other than 0 or 1. On the other hand, subjectivist approaches appear to be saddled with ‘coherence’ constraints on rational probabilities that require rational agents (...) to assign extremal probabilities to logical and mathematical propositions. In the light of these problems, many philosophers have come to think that there must be some way to generalize a Bayesian statistical account. In this article I propose that a classical frequentist approach should be reconsidered. I conclude that we can give a conditional justification of statistical testing of at least some mathematical hypotheses: if statistical tests provide us with reasons to believe or bet on empirical hypotheses in the standard situations, then they also provide us with reasons to believe or bet on mathematical hypotheses in the structurally similar mathematical cases. (shrink)

This paper is an extended review of the book Error and Inference, edited by Deborah Mayo and Aris Spanos, about their frequentist and philosophical perspective on testing of hypothesis and on the criticisms of alternatives like the Bayesian approach.

For model-based frequentist statistics, based on a parametric statistical model ${{\cal M}_\theta }$, the trustworthiness of the ensuing evidence depends crucially on the validity of the probabilistic assumptions comprising ${{\cal M}_\theta }$, the optimality of the inference procedures employed, and the adequateness of the sample size to learn from data by securing –. It is argued that the criticism of the postdata severity evaluation of testing results based on a small n by Rochefort-Maranda is meritless because it conflates (...) [a] misuses of testing with [b] genuine foundational problems. Interrogating this criticism reveals several misconceptions about trustworthy evidence and estimation-based effect sizes, which are uncritically embraced by the replication crisis literature. (shrink)

A key role in inference is played by randomization, which has been extensively used in clinical trials designs. Randomization is primarily intended to prevent the source of bias in treatment allocation by producing comparable groups. In the frequentist framework of inference, randomization allows also for the use of probability theory to express the likelihood of chance as a source for the difference of end outcome. In the Bayesian framework, its role is more nuanced. The Bayesian analysis of clinical trials (...) can afford a valid rationale for selective controls, pointing out a more limited role for randomization than it is generally accorded. This paper is aimed to offer a view of randomization from the perspective of both frequentist and Bayesian statistics and discussing the role of randomization also in theoretical decision models. (shrink)

In this paper epistemological, ontological and sociological questions concerning the statistical significance of sharp hypotheses in scientific research are investigated within the framework provided by Cognitive Constructivism and the FBST (Full Bayesian Significance Test). The constructivist framework is contrasted with the traditional epistemological settings for orthodox Bayesian and frequentist statistics provided by Decision Theory and Falsificationism.

The recent discovery of a Higgs boson prompted increased attention of statisticians and philosophers of science to the statistical methodology of High Energy Physics. Amidst long-standing debates within the field, HEP has adopted a mixed statistical methodology drawing upon both frequentist and Bayesian methods, but with standard frequentist techniques such as significance testing and confidence interval estimation playing a primary role. Physicists within HEP typically deny that their methodological decisions are guided by philosophical convictions, but are instead based (...) on “pragmatic” considerations, thus distancing themselves from what they perhaps perceive as an ongoing pitched ideological battle between frequentists and Bayesians. Here I argue that there is a philosophical orientation to HEP that is neither exclusively frequentist nor Bayesian, but that lies squarely in the tradition of philosophical pragmatism. I further argue that understanding the statistical methodology of HEP through the perspective of pragmatism clarifies the role of and rationale for significance testing in the search for new phenomena such as the Higgs boson. (shrink)

The history of statistics is filled with many controversies, in which the prime focus has been the difference in the “interpretation of probability” between Fre- quentist and Bayesian theories. Many philosophical arguments have been elabo- rated to examine the problems of both theories based on this dichotomized view of statistics, including the well-known stopping-rule problem and the catch-all hy- pothesis problem. However, there are also several “hybrid” approaches in theory, practice, and philosophical analysis. This poses many fundamental questions. (...) This paper reviews three cases and argues that the interpretation problem of probabil- ity is insufficient to begin a philosophical analysis of the current issues in the field of statistics. A novel viewpoint is proposed to examine the relationship between the stopping-rule problem and the catch-all hypothesis problem. (shrink)

The exponential growth of social data both in volume and complexity has increasingly exposed many of the shortcomings of the conventional frequentist approach to statistics. The scientific community has called for careful usage of the approach and its inference. Meanwhile, the alternative method, Bayesian statistics, still faces considerable barriers toward a more widespread application. The bayesvl R package is an open program, designed for implementing Bayesian modeling and analysis using the Stan language’s no-U-turn (NUTS) sampler. The package (...) combines the ability to construct Bayesian network models using directed acyclic graphs (DAGs), the Markov chain Monte Carlo (MCMC) simulation technique, and the graphic capability of the ggplot2 package. As a result, it can improve the user experience and intuitive understanding when constructing and analyzing Bayesian network models. A case example is offered to illustrate the usefulness of the package for Big Data analytics and cognitive computing. (shrink)

ABSTRACTIn this article, we discuss the benefits of Bayesian statistics and how to utilize them in studies of moral education. To demonstrate concrete examples of the applications of Bayesian statistics to studies of moral education, we reanalyzed two data sets previously collected: one small data set collected from a moral educational intervention experiment, and one big data set from a large-scale Defining Issues Test-2 survey. The results suggest that Bayesian analysis of data sets collected from moral educational studies (...) can provide additional useful statistical information, particularly that associated with the strength of evidence supporting alternative hypotheses, which has not been provided by the classical frequentist approach focusing on P-values. Finally, we introduce several practical guidelines pertaining to how to utilize Bayesian statistics, including the utilization of newly developed free statistical software, Jeffrey’s Amazing Statistics Program, and thresholding based on Bayes Factors, to scholars in the field of moral education. (shrink)

Statistical model specification and validation raise crucial foundational problems whose pertinent resolution holds the key to learning from data by securing the reliability of frequentist inference. The paper questions the judiciousness of several current practices, including the theory-driven approach, and the Akaike-type model selection procedures, arguing that they often lead to unreliable inferences. This is primarily due to the fact that goodness-of-fit/prediction measures and other substantive and pragmatic criteria are of questionable value when the estimated model is statistically misspecified. (...) Foisting one's favorite model on the data often yields estimated models which are both statistically and substantively misspecified, but one has no way to delineate between the two sources of error and apportion blame. The paper argues that the error statistical approach can address this Duhemian ambiguity by distinguishing between statistical and substantive premises and viewing empirical modeling in a piecemeal way with a view to delineate the various issues more effectively. It is also argued that Hendry's general to specific procedures does a much better job in model selection than the theory-driven and the Akaike-type procedures primary because of its error statistical underpinnings. (shrink)

Statisticians in medicine can disagree on appropriate methodology applicable to the design and analysis of clinical trials. So called Bayesians and frequentists both claim ethical superiority. This paper, by defining and then linking together various dichotomies, argues there is a place for both statistical camps. The choice between them depends on the phase of clinical trial, disease prevalence and severity, but supremely on the ethics underlying the particular trial. There is always a tension present between physicians primarily obligated to their (...) own patients (the weight of 'individual ethics') and ethical committees responsible for the scientific merit of the trial and its long-term implications ('collective ethics'). Individual ethics, it is proposed, favour the Bayesian approach; collective ethics, the frequentist. Though in some situations the choice appears clear-cut, there remain other where both methodologies can be appropriate. (shrink)

Increasingly, clinical research is evaluated on the quality of its statistical analysis. Traditionally, statistical analyses in clinical research have been carried out from a ‘frequentist’ perspective. The presence of an alternative paradigm – the Bayesian paradigm – has been relatively unknown in clinical research until recently. There is currently a growing interest in the use of Bayesian statistics in health care research. This is due both to a growing realization of the limitations of frequentist methods and to (...) the ability of Bayesian methods explicitly to incorporate prior expert knowledge and belief into the analyses. This is in contrast to frequentist methods, where prior experience and beliefs tend to be incorporated into the analyses in an ad hoc fashion. This paper outlines the frequentist and Bayesian paradigms. Acute myocardial infarction mortality data are then analysed from both a Bayesian and a frequentist perspective. In some analyses, the two methods are seen to produce comparable results; in others, they produce different results. It is noted that in this example, there are clinically relevant questions that are more easily addressed from a Bayesian perspective. Finally, areas in clinical research where Bayesian ideas are increasingly common are highlighted. (shrink)

This paper outlines the approaches of two apparently competing schools of statistics. The criticisms made by supporters of Bayesian statistics about conventional Frequentist statistics are explained, and the Bayesian claim that their method enables research into new treatments without the need for clinical trials is examined in detail. Several further important issues are considered, including: the use of historical controls and data routinely collected on patients; balance in randomised trials; the possibility of giving information to patients; (...) patient choice and patient autonomy; and how widely the results of clinical trials can be used. It is concluded that good statistical techniques in the design and analysis of medical studies are essential, but the statistical school used in developing such techniques is relatively unimportant. (shrink)

The data analyzed in this paper are part of the results described in Bueno et al. (2000). Three cytogenetics endpoints were analyzed in three populations of a species of wild rodent – Akodon montensis – living in an industrial, an agricultural, and a preservation area at the Itajaí Valley, State of Santa Catarina, Brazil. The polychromatic/normochromatic ratio, the mitotic index, and the frequency of micronucleated polychromatic erythrocites were used in an attempt to establish a genotoxic profile of each area. It (...) was assumed that the three populations were in the same conditions with respect to the influence of confounding factors such as animal age, health, nutrition status, presence of pathogens, and intra- and inter-populational genetic variability. Therefore, any differences found in the endpoints analyzed could be attributed to the external agents present in each area. The statistical models used in this paper are mixtures of negative-binomials and Poisson variables. The Poisson variables are used as approximations of binomials for rare events. The mixing distributions are beta densities. The statistical analyzes are under the bayesian perspective, as opposed to the frequentist ones often considered in the literature, as for instance in Bueno et al. (2000). (shrink)

Advocates of the self-corrective thesis argue that scientific method will refute false theories and find closer approximations to the truth in the long run. I discuss a contemporary interpretation of this thesis in terms of frequentist statistics in the context of the behavioral sciences. First, I identify experimental replications and systematic aggregation of evidence (meta-analysis) as the self-corrective mechanism. Then, I present a computer simulation study of scientific communities that implement this mechanism to argue that frequentist (...) class='Hi'>statistics may converge upon a correct estimate or not depending on the social structure of the community that uses it. Based on this study, I argue that methodological explanations of the “replicability crisis” in psychology are limited and propose an alternative explanation in terms of biases. Finally, I conclude suggesting that scientific self-correction should be understood as an interaction effect between inference methods and social structures. (shrink)

The p value is the probability under the null hypothesis of obtaining an experimental result that is at least as extreme as the one that we have actually obtained. That probability plays a crucial role in frequentist statistical inferences. But if we take the word ‘extreme’ to mean ‘improbable’, then we can show that this type of inference can be very problematic. In this paper, I argue that it is a mistake to make such an interpretation. Under minimal assumptions (...) about the alternative hypothesis, I explain why ‘extreme’ means ‘outside the most precise predicted range of experimental outcomes for a given upper bound probability of error’. Doing so, I rebut recent formulations of recurrent criticisms against the frequentist approach in statistics and underscore the importance of random variables. (shrink)

The way metrologists conceive of measurement has undergone a major shift in the last two decades. This shift can in great part be traced to a change in the statistical methods used to deal with the expression of measurement results, and, more particularly, with the calculation of measurement uncertainties. Indeed, as we show, the incapacity of the frequentist approach to the calculus of uncertainty to deal with systematic errors has prompted the replacement of the customary frequentist methods by (...) fully Bayesian procedures. The epistemological ramifications of the Bayesian approach merge with a deep empiricist mood tantamount to an “epistemic turn”: measurement results are analysed in terms of degrees of belief, and central concepts such as error and accuracy are called into question. We challenge the perspective entailed by this epistemic turn: we insist on the centrality of the concepts of error and accuracy by underlining the intentional character of measurement that is intimately linked to the process of correction of experimental data. We further circumvent the difficulties posed by the classical analysis of measurement by stressing the social rather than the epistemic dimension of measurement activities. (shrink)

Frequentist statistical methods continue to predominate in many areas of science despite prominent calls for "statistical reform." They do so in part because their main rivals, Bayesian methods, appeal to prior probability distributions that arguably lack an objective justification in typical cases. Some methodologists find a third approach called likelihoodism attractive because it avoids important objections to frequentism without appealing to prior probabilities. However, likelihoodist methods do not provide guidance for belief or action, but only assessments of data as (...) evidence. I argue that there is no good way to use those assessments to guide beliefs or actions without appealing to prior probabilities, and that as a result likelihoodism is not a viable alternative to frequentism and Bayesianism for statistical reform efforts in science. (shrink)

This article explores some open questions related to the problem of verification of theories in the context of empirical sciences by contrasting three epistemological frameworks. Each of these epistemological frameworks is based on a corresponding central metaphor, namely: (a) Neo-empiricism and the gambling metaphor; (b) Popperian falsificationism and the scientific tribunal metaphor; (c) Cognitive constructivism and the object as eigen-solution metaphor. Each of one of these epistemological frameworks has also historically co-evolved with a certain statistical theory and method for testing (...) scientific hypotheses, respectively: (a) Decision theoretic Bayesian statistics and Bayes factors; (b) Frequentist statistics and p-values; (c) Constructive Bayesian statistics and e-values. This article examines with special care the Zero Probability Paradox (ZPP), related to the verification of sharp or precise hypotheses. Finally, this article makes some remarks on Lakatos’ view of mathematics as a quasi-empirical science. (shrink)

In recent years, several remarkable results have shown that certain theorems of finite combinatorics are unprovable in certain logical systems. These developments have been instrumental in stimulating research in both areas, with the interface between logic and combinatorics being especially important because of its relation to crucial issues in the foundations of mathematics which were raised by the work of Kurt Godel. Because of the diversity of the lines of research that have begun to shed light on these issues, there (...) was a need for a comprehensive overview which would tie the lines together. This volume fills that need by presenting a balanced mixture of high quality expository and research articles that were presented at the August 1985 AMS-IMS-SIAM Joint Summer Research Conference, held at Humboldt State University in Arcata, California.With an introductory survey to put the works into an appropriate context, the collection consists of papers dealing with various aspects of 'unprovable theorems and fast-growing functions'. Among the topics addressed are: ordinal notations, the dynamical systems approach to Ramsey theory, Hindman's finite sums theorem and related ultrafilters, well quasiordering theory, uncountable combinatorics, nonstandard models of set theory, and a length-of-proof analysis of Godel's incompleteness theorem. Many of the articles bring the reader to the frontiers of research in this area, and most assume familiarity with combinatorics and/or mathematical logic only at the senior undergraduate or first-year graduate level. (shrink)

The likelihood principle of Bayesian statistics implies that information about the stopping rule used to collect evidence does not enter into the statistical analysis. This consequence confers an apparent advantage on Bayesian statistics over frequentist statistics. In the present paper, I argue that information about the stopping rule is nevertheless of value for an assessment of the reliability of the experiment, which is a pre-experimental measure of how well a contemplated procedure is expected to discriminate between (...) hypotheses. I show that, when reliability assessments enter into inquiries, some stopping rules prescribing optional stopping are unacceptable to both Bayesians and frequentists. (shrink)

The literature on thought experiments has been mainly concernedwith thought experiments that are directed at a theory, be it in aconstructive or a destructive manner. This has led somephilosophers to argue that all thought experiments can beformulated as arguments. The aim of this paper is to drawattention to a type of thought experiment that is not directed ata theory, but fulfills a specific function within a theory. Suchthought experiments are referred to as functional thoughtexperiments, and they are routinely used in (...) applied statistics. An example is given from frequentist statistics, where a thoughtexperiment is required to establish the probability space. It isconcluded that (a) not all thought experiments can be formulated asarguments, and (b) the role of thought experiments is more generaland more important to scientific reasoning than has previouslybeen recognized. (shrink)

I present a new proof of the likelihood principle that avoids two responses to a well-known proof due to Birnbaum ([1962]). I also respond to arguments that Birnbaum’s proof is fallacious, which if correct could be adapted to this new proof. On the other hand, I urge caution in interpreting proofs of the likelihood principle as arguments against the use of frequentist statistical methods. 1 Introduction2 The New Proof3 How the New Proof Addresses Proposals to Restrict Birnbaum’s Premises4 A (...) Response to Arguments that the Proofs Are Fallacious5 Conclusion. (shrink)

Conventional discussion of research methodology contrast two approaches, the quantitative and the qualitative, presented as collectively exhaustive. But if qualitative is taken as the understanding of lifeworlds, the two approaches between them cover only a tiny fraction of research methodologies; and the quantitative, taken as the routine application to controlled experiments of frequentist statistics by way of the Null Hypothesis Significance Testing Procedure, is seriously flawed. It is contrary to the advice both of Fisher and of Neyman and (...) Pearson, the two sources from which it is drawn. (shrink)