I discuss the philosophical significance of the statistical model selection criteria, in particular their relevance for philosophical of underdetermination. I present an easily comprehensible account of their simplest possible application and contrast it with their application to curve-fitting problems. I embed philosophers' earlier discussion concerning the situations in which the criteria yield implausible results into a more general framework. Among other things, I discuss a difficulty which is related to the so-called subfamily problem, and I show that it has analogies (...) in all legitimate applications of the model selection criteria, and that an analogy of Goodman's new riddle of induction can be formulated in only some of their applications. (shrink)

an observation to formulate a theory, it is no surprise that the resulting theory accurately captures that observation. However, when the theory makes a novel prediction—when it predicts an observation that was not used in its formulation—this seems to provide more substantial confirmation of the theory. This paper presents a new approach to the vexed problem of understanding the epistemic difference between prediction and accommodation. In fact, there are several problems that need to be disentangled; in all of them, the (...) key is the concept of overfitting. We float the hypothesis that accommodation is a defective methodology only when the methods used to accommodate the data fail to guard against the risk of overfitting. We connect our analysis with the proposals that other philosophers have made. We also discuss its bearing on the conflict between instrumentalism and scientific realism. Introduction Predictivisms—a taxonomy Observations Formulating the problem What might Annie be doing wrong? Solutions Observations explained Mayo on severe tests The miracle argument and scientific realism Concluding comments. (shrink)

The theory of fast and frugal heuristics, developed in a new book called Simple Heuristics that make Us Smart (Gigerenzer, Todd, and the ABC Research Group, in press), includes two requirements for rational decision making. One is that decision rules are bounded in their rationality –- that rules are frugal in what they take into account, and therefore fast in their operation. The second is that the rules are ecologically adapted to the environment, which means that they `fit to reality.' (...) The main purpose of this article is to apply these ideas to learning rules–-methods for constructing, selecting, or evaluating competing hypotheses in science, and to the methodology of machine learning, of which connectionist learning is a special case. The bad news is that ecological validity is particularly difficult to implement and difficult to understand. The good news is that it builds an important bridge from normative psychology and machine learning to recent work in the philosophy of science, which considers predictive accuracy to be a primary goal of science. (shrink)

The advent of formal definitions of the simplicity of a theory has important implications for model selection. But what is the best way to define simplicity? Forster and Sober ([1994]) advocate the use of Akaike's Information Criterion (AIC), a non-Bayesian formalisation of the notion of simplicity. This forms an important part of their wider attack on Bayesianism in the philosophy of science. We defend a Bayesian alternative: the simplicity of a theory is to be characterised in terms of Wallace's Minimum (...) Message Length (MML). We show that AIC is inadequate for many statistical problems where MML performs well. Whereas MML is always defined, AIC can be undefined. Whereas MML is not known ever to be statistically inconsistent, AIC can be. Even when defined and consistent, AIC performs worse than MML on small sample sizes. MML is statistically invariant under 1-to-1 re-parametrisation, thus avoiding a common criticism of Bayesian approaches. We also show that MML provides answers to many of Forster's objections to Bayesianism. Hence an important part of the attack on Bayesianism fails. (shrink)

The reliability challenge to ontology can be summarized as the complaint that no satisfying explanation is available of how one can have true ontological beliefs, given that the relevant belief-forming methods are noncausal (for example, not perception based or memory based). This paper first presents a version of the reliability challenge against realist approaches to ontology, put forward by Jared Warren. It then explores a response to the challenge on behalf of the realist that appeals to the use of abduction. (...) This response does not satisfactorily deal with the reliability challenge, though, and even leads to a further epistemic impasse. At this point, a version of ontological pluralism is presented, according to which all the competing theories in a certain ontological dispute can be true—in a sense of “true” to be articulated. The final step shows how this version of pluralism deals with the reliability challenge, especially with the complaint that we lack an explanation for our true ontological beliefs. (shrink)

Some think that issues to do with scientific method are last century's stale debate; Popper was an advocate of methodology, but Kuhn, Feyerabend, and others are alleged to have brought the debate about its status to an end. The papers in this volume show that issues in methodology are still very much alive. Some of the papers reinvestigate issues in the debate over methodology, while others set out new ways in which the debate has developed in the last decade. The (...) book will be of interest to philosophers and scientists alike in the reassessment it provides of earlier debates about method and current directions of research. (shrink)