No one has a well developed solution to Duhem's problem, the problem of how experimental evidence warrants revision of our theories. Deborah Mayo proposes a solution to Duhem's problem in route to her more ambitious program of providing a philosophical account of inductive inference and experimental knowledge. This paper is a response to Mayo's Error Statistics (ES) program, paying particular attention to her response to Duhem's problem. It turns out that Mayo's purported solution to Duhem's problem is very significant to (...) her project, for the epistemic license claimed by ES and the philosophical underpinnings to her account of experimental knowledge depend on this solution. By introducing the partition problem, I argue that ES fails to solve Duhem's problem and therefore fails to provide an adequate account of experimental knowledge. (shrink)

Disputes between advocates of Bayesians and more orthodox approaches to statistical inference presuppose that Bayesians must regard must regard stopping rules, which play an important role in orthodox statistical methods, as evidentially irrelevant.In this essay, I show that this is not the case and that the stopping rule is evidentially relevant given some Bayesian confirmation measures that have been seriously proposed. However, I show that accepting a confirmation measure of this sort comes at the cost of rejecting two useful ancillaryBayesian (...) principles. (shrink)

This article calls into question the charge that frequentist testing is susceptible to the base-rate fallacy. It is argued that the apparent similarity between examples like the Harvard Medical School test and frequentist testing is highly misleading. A closer scrutiny reveals that such examples have none of the basic features of a proper frequentist test, such as legitimate data, hypotheses, test statistics, and sampling distributions. Indeed, the relevant error probabilities are replaced with the false positive/negative rates that constitute deductive calculations (...) based on known probabilities among events. As a result, the ampliative dimension of frequentist induction—learning from data about the underlying data-generating mechanism—is missing. *Received August 2009; revised January 2010. †To contact the author, please write to: Department of Economics, Virginia Tech, Blacksburg, VA 24061; e-mail: [email protected]. (shrink)

In the first section, five major attempts to solve the problem of induction and their failures are discussed. In the second section, an account of meta-induction is introduced. It offers a novel solution to the problem of induction, based on mathematical theorems about the predictive optimality of attractivity-weighted meta-induction. In the third section, how the a priori justification of meta-induction provides a non-circular a posteriori justification of object-induction, based on its superior track record, is explained. In the fourth section, four (...) important extensions and refinements of the method of meta-induction are presented. The final section, summarizes the major impacts of the program of meta-induction for epistemology, the philosophy of science and cognitive science. (shrink)

This paper describes the corner-stones of a means-ends approach to the philosophy of inductive inference. I begin with a fallibilist ideal of convergence to the truth in the long run, or in the 'limit of inquiry'. I determine which methods are optimal for attaining additional epistemic aims (notably fast and steady convergence to the truth). Means-ends vindications of (a version of) Occam's Razor and the natural generalizations in a Goodmanian Riddle of Induction illustrate the power of this approach. The paper (...) establishes a hierarchy of means-ends notions of empirical success, and discusses a number of issues, results and applications of means-ends epistemology. (shrink)

: Bayesians and error statisticians have relied heavily upon examples from physics in developing their accounts of scientific inference. The present essay demonstrates it is possible to analyze H.B.D. Kettlewell's classic study of natural selection from Deborah Mayo's error statistical point of view (Mayo 1996). A comparison with a previous analysis of this episode from a Bayesian perspective (Rudge 1998) reveals that the error statistical account makes better sense of investigations such as Kettlewell's because it clarifies how core elements in (...) the design of experiments are used to minimize erroneous inferences rather than dwelling on whether the strategies used are reasonable. (shrink)

Despite the widespread use of key concepts of the Neyman–Pearson (N–P) statistical paradigm—type I and II errors, significance levels, power, confidence levels—they have been the subject of philosophical controversy and debate for over 60 years. Both current and long-standing problems of N–P tests stem from unclarity and confusion, even among N–P adherents, as to how a test's (pre-data) error probabilities are to be used for (post-data) inductive inference as opposed to inductive behavior. We argue that the relevance of error probabilities (...) is to ensure that only statistical hypotheses that have passed severe or probative tests are inferred from the data. The severity criterion supplies a meta-statistical principle for evaluating proposed statistical inferences, avoiding classic fallacies from tests that are overly sensitive, as well as those not sensitive enough to particular errors and discrepancies. Introduction and overview 1.1 Behavioristic and inferential rationales for Neyman–Pearson (N–P) tests 1.2 Severity rationale: induction as severe testing 1.3 Severity as a meta-statistical concept: three required restrictions on the N–P paradigm Error statistical tests from the severity perspective 2.1 N–P test T(): type I, II error probabilities and power 2.2 Specifying test T() using p-values Neyman's post-data use of power 3.1 Neyman: does failure to reject H warrant confirming H? Severe testing as a basic concept for an adequate post-data inference 4.1 The severity interpretation of acceptance (SIA) for test T() 4.2 The fallacy of acceptance (i.e., an insignificant difference): Ms Rosy 4.3 Severity and power Fallacy of rejection: statistical vs. substantive significance 5.1 Taking a rejection of H0 as evidence for a substantive claim or theory 5.2 A statistically significant difference from H0 may fail to indicate a substantively important magnitude 5.3 Principle for the severity interpretation of a rejection (SIR) 5.4 Comparing significant results with different sample sizes in T(): large n problem 5.5 General testing rules for T(), using the severe testing concept The severe testing concept and confidence intervals 6.1 Dualities between one and two-sided intervals and tests 6.2 Avoiding shortcomings of confidence intervals Beyond the N–P paradigm: pure significance, and misspecification tests Concluding comments: have we shown severity to be a basic concept in a N–P philosophy of induction? (shrink)

Science depends on judgments of the bearing of evidence on theory. Scientists must judge whether an observation or the result of an experiment supports, disconfirms, or is simply irrelevant to a given hypothesis. Similarly, scientists may judge that, given all the available evidence, a hypothesis ought to be accepted as correct or nearly so, rejected as false, or neither. Occasionally, these evidential judgments can be made on deductive grounds. If an experimental result strictly contradicts a hypothesis, then the truth of (...) the data deductively entails the falsity of the hypothesis. In the great majority of cases, however, the connection between evidence and hypothesis is non-demonstrative, or inductive. In particular, this is so whenever a general hypothesis is inferred to be correct on the basis of the available data, since the truth of the data will not deductively entail the truth of the hypothesis. It always remains possible that the hypothesis is false even though the data are correct. (shrink)

Economists reason causally. Like many other scientists, they aim at formulating justified causal claims about their object of study. This thesis contributes to our understanding of how causal reasoning proceeds in economics. By using the research on the causes of unemployment as a case study, three questions are adressed. What are the meanings of causal claims? How can a causal claim be adequately supported by evidence? How are causal beliefs affected by incoming facts? In the process of answering these semantic, (...) epistemic and dynamical questions, the richness of causal reasoning in economics is brought to the foreground. In the part on semantics, it is argued that the meaning of causal generalizations in policy-oriented economics is best captured by using the concept of ‘inferential relation’ rather than the one of ‘referential relation’. This part contributes to develop an inferentialist semantics by distinguishing among types of inferential relations. In the part on epistemology, it is argued that justification in sciences like economics often relies, and ought to rely, on evidential variety―i.e. the combination of evidence from multiple sources. An interpretation of independence among evidential sources is offered: two evidential sources are fully independent if and only if the reasons why they would be unreliable do not overlap. It is then shown that the variety-of-evidence thesis does not hold under extreme circumstances. The part on dynamics focuses on deviant-case study. A storyline is provided for what deviant-case study aims at and how it proceeds in what is characterized as an ‘eclectic science’. (shrink)

Philosophers of science turned to historical case studies in part in response to Thomas Kuhn's insistence that such studies can transform the philosophy of science. In this issue Joseph Pitt argues that the power of case studies to instruct us about scientific methodology and epistemology depends on prior philosophical commitments, without which case studies are not philosophically useful. Here I reply to Pitt, demonstrating that case studies, properly deployed, illustrate styles of scientific work and modes of argumentation that are not (...) well handled by currently standard philosophical analyses. I illustrate these claims with exemplary findings from case studies dealing with exploratory experimentation and with interdisciplinary cooperation across sciences to yield multiple independent means of access to theoretical entities. The latter cases provide examples of ways that scientists support claims about theoretical entities that are not available in work performed within a single discipline. They also illustrate means of correcting systematic biases that stem from the commitments of each discipline taken separately. These findings illustrate the transformative power of case study methods, allow us to escape from the horns of Pitt's "dilemma of case studies," and vindicate some of the post-Kuhn uses to which case studies have been put. (shrink)

The notion of a severe test has played an important methodological role in the history of science. But it has not until recently been analyzed in any detail. We develop a generally Bayesian analysis of the notion, compare it with Deborah Mayo’s error-statistical approach by way of sample diagnostic tests in the medical sciences, and consider various objections to both. At the core of our analysis is a distinction between evidence and confirmation or belief. These notions must be kept separate (...) if mistakes are to be avoided; combined in the right way, they provide an adequate understanding of severity. Those who think that the weight of the evidence always enables you to choose between hypotheses “ignore one of the factors (the prior probability) altogether, and treat the other (the likelihood) as though it ...meant something other than it actually does. This is the same mistake as is made by someone who has scruples about measuring the arms of a balance (having only a tape measure at his disposal ...), but is willing to assert that the heavier load will always tilt the balance (thereby implicitly assuming, although without admitting it, that the arms are of equal length!). (Bruno de Finetti, Theory of Probability)2. (shrink)

This thesis has two goals. Firstly, we consider the problem of model selection for the purposes of prediction. In modern science predictive mathematical models are ubiquitous and can be found in such diverse fields as weather forecasting, economics, ecology, mathematical psychology, sociology, etc. It is often the case that for a given domain of inquiry there are several plausible models, and the issue then is how to discriminate between them – this is the problem of model selection. We consider approaches (...) to model selection that are used in classical [also known as frequentist] statistics, and fashionable in recent years methods of Akaike Information Criterion [AIC] and Bayes Information Criterion [BIC], the latter being a part of a broader Bayesian approach. We show the connection between AIC and BIC, and provide comparison of performance of these methods. Secondly, we consider some philosophical arguments that arise within the setting of the model selection approaches investigated in the first part. These arguments aim to provide counterexamples to the epistemic thesis of scientific realism, viz., that predictively successful scientific theories are approximately true, and to the idea that truth and predictive accuracy go together. We argue for the following claims: 1) that none of the criticisms brought forward in the philosophical literature against the AIC methodology are devastating, and AIC remains a viable method of model selection; 2) that the BIC methodology likewise survives the numerous criticisms; 3) that the counterexamples to scientific realism that ostensibly arise within the framework of model selection are flawed; 4) that in general the model selection methods discussed in this thesis are neutral with regards to the issue of scientific realism; 5) that a plurality of methodologies should be applied to the problem of model selection with full awareness of the foundational issues that each of these methodologies has. (shrink)

Proponents of Bayesian confirmation theory believe that they have the solution to a significant, recalcitrant problem in philosophy of science. It is the identification of the logic that governs evidence and its inductive bearing in science. That is the logic that lets us say that our catalog of planetary observations strongly confirms Copernicus’ heliocentric hypothesis; or that the fossil record is good evidence for the theory of evolution; or that the 3oK cosmic background radiation supports big bang cosmology. The definitive (...) solution to this problem would be a significant achievement. The problem is of central importance to philosophy of science, for, in the end, what distinguishes science from myth making is that we have good evidence for the content of science, or at least of mature sciences, whereas myths are evidentially ungrounded fictions. The core ideas shared by all versions of Bayesian confirmation theory are, at a good first approximation, that a scientist’s beliefs are or should conform to a probability measure; and that the incorporation of new evidence is through conditionalization using Bayes’ theorem. While the burden of this chapter will be to inventory why critics believe this theory may not be the solution after all, it is worthwhile first to summarize here the most appealing virtues of this simple account. There are three. First, the theory reduces the often nebulous notion of a logic of.. (shrink)

Philosophical theories of evidence have been on offer, but they are mostly evaluated in terms of all-or-none desiderata — if they fail to meet one of the desiderata, they are not a satisfactory theory. In this thesis, I aim to accomplish three missions. Firstly, I construct a new way of evaluating theories of evidence, which I call a tool-based view. Secondly, I analyse the nature of what I will call the various relevance-mediating vehicles that each theory of evidence employs. Thirdly, (...) I articulate the comparative core of evidential reasoning in the historical sciences, one which is overlooked in major theories of evidence. On the first mission, I endorse a meta-thesis of pluralism on theories of evidence, namely a tool-based view. I regard a theory of evidence as a purpose-specific and setting-sensitive tool which has its own strengths, difficulties and limitations. Among the major theories of evidence I have reviewed, I focus on Achinstein’s explanationist theory, Cartwright’s argument theory and Reiss’s inferentialist account, scrutinising and evaluating them against the purposes they set out and the scope of their applications. On the second mission, I note that there is no such thing as intrinsically ‘being evidence’. Rather, I hold that relevance-mediating vehicles configure data, materials or claims in such ways that some of them are labelled evidence. I identify the relevance-mediating vehicles that the theories of evidence employ. On the final mission, I argue that the likelihoodist account is an appropriate tool for explaining the evidential reasoning in poorly specified settings where likelihoods can be only imprecisely compared. Such settings, I believe, are typical in the historical sciences. Using the reconstruction of proto-sounds in historical linguistics as a case study, I formalise the rationale behind it by means of the law of likelihood. (shrink)