Unsupervised learning algorithms are widely used for many important statistical tasks with numerous applications in science and industry. Yet despite their prevalence, they have attracted remarkably little philosophical scrutiny to date. This stands in stark contrast to supervised and reinforcement learning algorithms, which have been widely studied and critically evaluated, often with an emphasis on ethical concerns. In this article, I analyze three canonical unsupervised learning problems: clustering, abstraction, and generative modeling. I argue that these methods raise unique epistemological and (...) ontological questions, providing data-driven tools for discovering natural kinds and distinguishing essence from contingency. This analysis goes some way toward filling the lacuna in contemporary philosophical discourse on unsupervised learning, as well as bringing conceptual unity to a heterogeneous field more often described by what it isnot(i.e., supervised or reinforcement learning) than by what itis. I submit that unsupervised learning is not just a legitimate subject of philosophical inquiry but perhaps the most fundamental branch of all AI. However, an uncritical overreliance on unsupervised methods poses major epistemic and ethical risks. I conclude by advocating for a pragmatic, error-statistical approach that embraces the opportunities and mitigates the challenges posed by this powerful class of algorithms. (shrink)

The main aim of this paper is to revisit the curve fitting problem using the reliability of inductive inference as a primary criterion for the ‘fittest' curve. Viewed from this perspective, it is argued that a crucial concern with the current framework for addressing the curve fitting problem is, on the one hand, the undue influence of the mathematical approximation perspective, and on the other, the insufficient attention paid to the statistical modeling aspects of the problem. Using goodness-of-fit as the (...) primary criterion for ‘best', the mathematical approximation perspective undermines the reliability of inference objective by giving rise to selection rules which pay insufficient attention to ‘accounting for the regularities in the data'. A more appropriate framework is offered by the error-statistical approach, where (i) statistical adequacy provides the criterion for assessing when a curve captures the regularities in the data adequately, and (ii) the relevant error probabilities can be used to assess the reliability of inductive inference. Broadly speaking, the fittest curve (statistically adequate) is not determined by the smallness if its residuals, tempered by simplicity or other pragmatic criteria, but by the nonsystematic (e.g. white noise) nature of its residuals. The advocated error-statistical arguments are illustrated by comparing the Kepler and Ptolemaic models on empirical grounds. ‡I am grateful to Deborah Mayo and Clark Glymour for many valuable suggestions and comments on an earlier draft of the paper; estimating the Ptolemaic model was the result of Glymour's prompting and encouragement. †To contact the author, please write to: Department of Economics, Virginia Tech 3019 Pamplin Hall (0316), Blacksburg, VA 24061; e-mail: [email protected]. (shrink)

The Akaike Information Criterion can be a valuable tool of scientific inference. This statistic, or any other statistical method for that matter, cannot, however, be the whole of scientific methodology. In this paper some of the limitations of Akaikean statistical methods are discussed. It is argued that the full import of empirical evidence is realized only by adopting a richer ideal of empirical success than predictive accuracy, and that the ability of a theory to turn phenomena into accurate, agreeing measurements (...) of causally relevant parameters contributes to the evidential support of the theory. This is illustrated by Newton's argument from orbital phenomena to the inverse-square law of gravitation. (shrink)

The Akaike Information Criterion can be a valuable tool of scientific inference. This statistic, or any other statistical method for that matter, cannot, however, be the whole of scientific methodology. In this paper some of the limitations of Akaikean statistical methods are discussed. It is argued that the full import of empirical evidence is realized only by adopting a richer ideal of empirical success than predictive accuracy, and that the ability of a theory to turn phenomena into accurate, agreeing measurements (...) of causally relevant parameters contributes to the evidential support of the theory. This is illustrated by Newton's argument from orbital phenomena to the inverse-square law of gravitation. (shrink)

Copernicus claimed that his system was preferable in part on the grounds of its superior harmony and simplicity, but left very few hints as to what was meant by these terms. Copernicus’s pupil, Rheticus, was more forthcoming. Kepler, influenced by Rheticus, articulated further the nature of the virtues of harmony and simplicity. I argue that these terms are metaphors for the structural features of the Copernican system that make it more able to effectively exploit the available data. So it is (...) a mistake to conclude that early Copernicans could only offer aesthetic or pragmatic arguments; they could and did offer evidential ones as well. Moreover, the evidential arguments they offered parallel current arguments in the philosophical literature.Keywords: Nicolaus Copernicus; Georg Joachim Rheticus; Johannes Kepler; Simplicity; Bayesianism; Hirotsugu Akaike. (shrink)

In this article I give a naturalistic-cum-formal analysis of therelation between beauty, empirical success, and truth. The analysis is based on the onehand on a hypothetical variant of the so-called `mere-exposure effect'' which has been more orless established in experimental psychology regarding exposure-affect relationshipsin general and aesthetic appreciation in particular (Zajonc 1968; Temme 1983; Bornstein 1989;Ye 2000). On the other hand it is based on the formal theory of truthlikeness andtruth approximation as presented in my From Instrumentalism to Constructive Realism (...) (2000).The analysis supports the findings of James McAllister in his beautifulBeauty and Revolution in Science (1996), by explaining and justifying them.First, scientists are essentially right in regarding aesthetic criteria useful for empiricalprogress and even for truth approximation, provided they conceive of them as less hard thanempirical criteria. Second, the aesthetic criteria of the time, the `aesthetic canon'', maywell be based on `aesthetic induction'' regarding nonempirical features of paradigms of successfultheories which scientists have come to appreciate as beautiful. Third, aestheticcriteria can play a crucial, schismatic role in scientific revolutions. Since they may well be wrong, they may, in the hands of aesthetic conservatives, retard empirical progress and hence truth approximation, but this does not happen in the hands of aesthetically flexible, `revolutionary'' scientists. We may find totallyopposite things beautiful: a simplemathematical principle as well as a series of unrepeatable complex contingencies. It is a matter of psychology.(Stephen Jay Gould, translated passage from (Kayzer 2000, 30)). (shrink)

Two Bayesian approaches to choosing between statistical models are contrasted. One of these is an approach which Bayesian statisticians regularly use for motivating the use of AIC, BIC, and other similar model selection criteria, and the other one is a new approach which has recently been proposed by Bandyopadhayay, Boik, and Basu. The latter approach is criticized, and the basic ideas of the former approach are presented in a way that makes them accessible to a philosophical audience. It is also (...) pointed out that the former approach establishes a new, philosophically interesting connection between the notions of simplicity and informativeness. (shrink)

What has science actually achieved? A theory of achievement should define what has been achieved, describe the means or methods used in science, and explain how such methods lead to such achievements. Predictive accuracy is one truth‐related achievement of science, and there is an explanation of why common scientific practices tend to increase predictive accuracy. Akaike’s explanation for the success of AIC is limited to interpolative predictive accuracy. But therein lies the strength of the general framework, for it also provides (...) a clear formulation of many open problems of research. (shrink)

What has science actually achieved? A theory of achievement should define what has been achieved, describe the means or methods used in science, and explain how such methods lead to such achievements. Predictive accuracy is one truth-related achievement of science, and there is an explanation of why common scientific practices tend to increase predictive accuracy. Akaike's explanation for the success of AIC is limited to interpolative predictive accuracy. But therein lies the strength of the general framework, for it also provides (...) a clear formulation of many open problems of research. (shrink)

Recent solutions to the curve-fitting problem, described in Forster and Sober ([1995]), trade off the simplicity and fit of hypotheses by defining simplicity as the paucity of adjustable parameters. Scott De Vito ([1997]) charges that these solutions are 'conventional' because he thinks that the number of adjustable parameters may change when the hypotheses are described differently. This he believes is exactly what is illustrated in Goodman's new riddle of induction, otherwise known as the grue problem. However, the 'number of adjustable (...) parameters' is actually a loose way of referring to a quantity that is not language dependent. The quantity arises out of Akaike's theorem in a way that ensures its language invariance. (shrink)

The theory of fast and frugal heuristics, developed in a new book called Simple Heuristics that make Us Smart (Gigerenzer, Todd, and the ABC Research Group, in press), includes two requirements for rational decision making. One is that decision rules are bounded in their rationality –- that rules are frugal in what they take into account, and therefore fast in their operation. The second is that the rules are ecologically adapted to the environment, which means that they `fit to reality.' (...) The main purpose of this article is to apply these ideas to learning rules–-methods for constructing, selecting, or evaluating competing hypotheses in science, and to the methodology of machine learning, of which connectionist learning is a special case. The bad news is that ecological validity is particularly difficult to implement and difficult to understand. The good news is that it builds an important bridge from normative psychology and machine learning to recent work in the philosophy of science, which considers predictive accuracy to be a primary goal of science. (shrink)

The simple question, what is empirical success? turns out to have a surprisingly complicated answer. We need to distinguish between meritorious fit and ‘fudged fit', which is akin to the distinction between prediction and accommodation. The final proposal is that empirical success emerges in a theory dependent way from the agreement of independent measurements of theoretically postulated quantities. Implications for realism and Bayesianism are discussed. ‡This paper was written when I was a visiting fellow at the Center for Philosophy of (...) Science at the University of Pittsburgh; I thank everyone for their support. †To contact the author, please write to: Department of Philosophy, University of Wisconsin–Madison, 5185 Helen C. White Hall, 600 North Park Street, Madison, WI 53706; e-mail: [email protected]. (shrink)

One of the main challenges confronting Humean accounts of natural law is that Humean laws appear to be unable to play the explanatory role of laws in scientific practice. The worry is roughly that if the laws are just regularities in the particular matters of fact (as the Humean would have it), then they cannot also explain the particular matters of fact, on pain of circularity. Loewer (2012) has defended Humeanism, arguing that this worry only arises if we fail to (...) distinguish between scientific and metaphysical explanations. However, Lange (2013, 2018) has argued that scientific and metaphysical explanations are linked by a transitivity principle, which would undercut Loewer's defense and re-ignite the circularity worry for the Humean. I argue here that the Humean has antecedent reasons to doubt that there are any systematic connections between scientific and metaphysical explanations. The reason is that the Humean should think that scientific and metaphysical explanation have disparate aims, and therefore that neither form of explanation is beholden to the other in its pronouncements about what explains what. Consequently, the Humean has every reason to doubt that Lange's transitivity principle obtains. (shrink)

In the curve fitting problem two conflicting desiderata, simplicity and goodness-of-fit pull in opposite directions. To solve this problem, two proposals, the first one based on Bayes's theorem criterion (BTC) and the second one advocated by Forster and Sober based on Akaike's Information Criterion (AIC) are discussed. We show that AIC, which is frequentist in spirit, is logically equivalent to BTC, provided that a suitable choice of priors is made. We evaluate the charges against Bayesianism and contend that AIC approach (...) has shortcomings. We also discuss the relationship between Schwarz's Bayesian Information Criterion and BTC. (shrink)

This thesis has two goals. Firstly, we consider the problem of model selection for the purposes of prediction. In modern science predictive mathematical models are ubiquitous and can be found in such diverse fields as weather forecasting, economics, ecology, mathematical psychology, sociology, etc. It is often the case that for a given domain of inquiry there are several plausible models, and the issue then is how to discriminate between them – this is the problem of model selection. We consider approaches (...) to model selection that are used in classical [also known as frequentist] statistics, and fashionable in recent years methods of Akaike Information Criterion [AIC] and Bayes Information Criterion [BIC], the latter being a part of a broader Bayesian approach. We show the connection between AIC and BIC, and provide comparison of performance of these methods. Secondly, we consider some philosophical arguments that arise within the setting of the model selection approaches investigated in the first part. These arguments aim to provide counterexamples to the epistemic thesis of scientific realism, viz., that predictively successful scientific theories are approximately true, and to the idea that truth and predictive accuracy go together. We argue for the following claims: 1) that none of the criticisms brought forward in the philosophical literature against the AIC methodology are devastating, and AIC remains a viable method of model selection; 2) that the BIC methodology likewise survives the numerous criticisms; 3) that the counterexamples to scientific realism that ostensibly arise within the framework of model selection are flawed; 4) that in general the model selection methods discussed in this thesis are neutral with regards to the issue of scientific realism; 5) that a plurality of methodologies should be applied to the problem of model selection with full awareness of the foundational issues that each of these methodologies has. (shrink)

It is a standard feature of the BSA and its variants that systematizations of the world competing to be the best must be expressed in the same language. This paper argues that such single language privileging is problematic because it enhances the objection that the BSA is insufficiently objective, and it breaks the parallel between the BSA and scientific practice by not letting laws and basic kinds be identified/discovered together. A solution to these problems and the ones that prompt single (...) language privileging is proposed in the form of privileging the best system competition. (shrink)

Probabilistic inference from frequencies, such as "Most Quakers are pacifists; Nixon is a Quaker, so probably Nixon is a pacifist" suffer from the problem that an individual is typically a member of many "reference classes" (such as Quakers, Republicans, Californians, etc) in which the frequency of the target attribute varies. How to choose the best class or combine the information? The article argues that the problem can be solved by the feature selection methods used in contemporary Big Data science: the (...) correct reference class is that determined by the features relevant to the target, and relevance is measured by correlation (that is, a feature is relevant if it makes a difference to the frequency of the target). (shrink)