Model-Selection Theory: The Need for a More Nuanced Picture of Use-Novelty and Double-Counting

British Journal for the Philosophy of Science:axw024 (2016)
  Copy   BIBTEX

Abstract

This article argues that common intuitions regarding (a) the specialness of ‘use-novel’ data for confirmation and (b) that this specialness implies the ‘no-double-counting rule’, which says that data used in ‘constructing’ (calibrating) a model cannot also play a role in confirming the model’s predictions, are too crude. The intuitions in question are pertinent in all the sciences, but we appeal to a climate science case study to illustrate what is at stake. Our strategy is to analyse the intuitive claims in light of prominent accounts of confirmation of model predictions. We show that on the Bayesian account of confirmation, and also on the standard classical hypothesis-testing account, claims (a) and (b) are not generally true; but for some select cases, it is possible to distinguish data used for calibration from use-novel data, where only the latter confirm. The more specialized classical model-selection methods, on the other hand, uphold a nuanced version of claim (a), but this comes apart from (b), which must be rejected in favour of a more refined account of the relationship between calibration and confirmation. Thus, depending on the framework of confirmation, either the scope or the simplicity of the intuitive position must be revised.

Similar books and articles

Climate models, calibration, and confirmation.Charlotte Werndl & Katie Steele - 2013 - British Journal for the Philosophy of Science 64 (3):609-635.
Climate Models, Calibration, and Confirmation.Katie Steele & Charlotte Werndl - 2013 - British Journal for the Philosophy of Science 64 (3):609-635.
Predictivism and old evidence: a critical look at climate model tuning.Mathias Frisch - 2015 - European Journal for Philosophy of Science 5 (2):171-190.
Model tuning in engineering: uncovering the logic.Katie Steele & Charlotte Werndl - 2015 - Journal of Strain Analysis for Engineering Design 51 (1):63-71.
The role of Bayesian philosophy within Bayesian model selection.Jan Sprenger - 2013 - European Journal for Philosophy of Science 3 (1):101-114.
Varieties of support and confirmation of climate models.Elisabeth A. Lloyd - 2009 - Aristotelian Society Supplementary Volume 83 (1):213-232.
Simplicity and model selection.Guillaume Rochefort-Maranda - 2016 - European Journal for Philosophy of Science 6 (2):261-279.
Confirmation of ecological and evolutionary models.Elisabeth A. Lloyd - 1987 - Biology and Philosophy 2 (3):277-293.
Bayesian Confirmation: A Means with No End.Peter Brössel & Franz Huber - 2015 - British Journal for the Philosophy of Science 66 (4):737-749.
The epistemology of climate models and some of its implications for climate science and the philosophy of science.Joel Katzav - 2014 - Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics 46 (2):228-238.
Model robustness as a confirmatory virtue: The case of climate science.Elisabeth A. Lloyd - 2015 - Studies in History and Philosophy of Science Part A 49:58-68.

Analytics

Added to PP
2015-12-01

Downloads
403 (#44,844)

6 months
53 (#70,477)

Historical graph of downloads
How can I increase my downloads?

Author Profiles

Katie Steele
Australian National University
Charlotte Werndl
University of Salzburg

Citations of this work

Models in Search of Targets: Exploratory Modelling and the Case of Turing Patterns.Axel Gelfert - 2018 - In A. Christian, David Hommen, N. Retzlaff & Gerhard Schurz (eds.), Philosophy of Science. European Studies in Philosophy of Science, vol 9. Springer International Publishing. pp. 245-269.
Predictivism and model selection.Alireza Fatollahi - 2023 - European Journal for Philosophy of Science 13 (1):1-28.

Add more citations

References found in this work

Prediction versus accommodation and the risk of overfitting.Christopher Hitchcock & Elliott Sober - 2004 - British Journal for the Philosophy of Science 55 (1):1-34.
Logical versus historical theories of confirmation.Alan Musgrave - 1974 - British Journal for the Philosophy of Science 25 (1):1-23.
Climate Models, Calibration, and Confirmation.Katie Steele & Charlotte Werndl - 2013 - British Journal for the Philosophy of Science 64 (3):609-635.
Novel evidence and severe tests.Deborah G. Mayo - 1991 - Philosophy of Science 58 (4):523-552.

View all 18 references / Add more references