Many examples of calibration in climate science raise no alarms regarding model reliability. We examine one example and show that, in employing Classical Hypothesis-testing, it involves calibrating a base model against data that is also used to confirm the model. This is counter to the "intuitive position". We argue, however, that aspects of the intuitive position are upheld by some methods, in particular, the general Cross-validation method. How Cross-validation relates to other prominent Classical methods such as the Akaike Information Criterion (...) and Bayesian Information Criterion is also discussed. (shrink)

The aim of the article is to provide a clear and thorough conceptual analysis of the main candidates for a definition of climate and climate change. Five desiderata on a definition of climate are presented: it should be empirically applicable; it should correctly classify different climates; it should not depend on our knowledge; it should be applicable to the past, present, and future; and it should be mathematically well-defined. Then five definitions are discussed: climate as distribution over time for constant (...) external conditions; climate as distribution over time when the external conditions vary as in reality; climate as distribution over time relative to regimes of varying external conditions; climate as the ensemble distribution for constant external conditions; and climate as the ensemble distribution when the external conditions vary as in reality. The third definition is novel and is introduced as a response to problems with existing definitions. The conclusion is that most definitions encounter serious problems and that the third definition is most promising. 1 Introduction2 Climate Variables and a Simple Climate Model3 Desiderata on a Definition of Climate4 Climate as Distribution over Time4.1 Definition 1: Distribution over time for constant external conditions4.2 Definition 2: Distribution over time when the external conditions vary as in reality4.3 Definition 3: Distribution over time for regimes of varying external conditions4.4 Infinite versions5 Climate as Ensemble Distribution5.1 Definition 4: Ensemble distribution for constant external conditions5.2 Definition 5: Ensemble distribution when the external conditions vary as in reality5.3 Infinite versions6 ConclusionAppendix. (shrink)

This article argues that common intuitions regarding (a) the specialness of ‘use-novel’ data for confirmation and (b) that this specialness implies the ‘no-double-counting rule’, which says that data used in ‘constructing’ (calibrating) a model cannot also play a role in confirming the model’s predictions, are too crude. The intuitions in question are pertinent in all the sciences, but we appeal to a climate science case study to illustrate what is at stake. Our strategy is to analyse the intuitive claims in (...) light of prominent accounts of confirmation of model predictions. We show that on the Bayesian account of confirmation, and also on the standard classical hypothesis-testing account, claims (a) and (b) are not generally true; but for some select cases, it is possible to distinguish data used for calibration from use-novel data, where only the latter confirm. The more specialized classical model-selection methods, on the other hand, uphold a nuanced version of claim (a), but this comes apart from (b), which must be rejected in favour of a more refined account of the relationship between calibration and confirmation. Thus, depending on the framework of confirmation, either the scope or the simplicity of the intuitive position must be revised. (shrink)

We give an overview of the practice of developing and using complex climate models, as seen from experiences in a major climate modelling center and through participation in the Coupled Model Intercomparison Project. We discuss the construction and calibration of models; their evaluation, especially through use of out-of-sample tests; and their exploitation in multi-model ensembles to identify biases and make predictions. We stress that adequacy or utility of climate models is best assessed via their skill against more naïve predictions. The (...) framework we use for making inferences about reality using simulations is naturally Bayesian, and has many points of contact with more familiar examples of scientific epistemology. While the use of complex simulations in science is a development that changes much in how science is done in practice, we argue that the concepts being applied fit very much into traditional practices of the scientific method, albeit those more often associated with laboratory work. (shrink)

I examine error diagnosis (model-model disagreement) in climate model intercomparisons including its difficulties, fruitful examples, and prospects for streamlining error diagnosis. I suggest that features of climate model intercomparisons pose a more significant challenge for error diagnosis than do features of individual model construction and complexity. Such features of intercomparisons include, e.g., the number of models involved, how models from different institutions interrelate, and what scientists know about each model. By considering numerous examples in the climate modeling literature, I distill (...) general strategies (e.g., employing physical reasoning and using dimension reduction techniques) used to diagnose model error. Based on these examples, I argue that an error repertoire could be beneficial for improving error diagnosis in climate modeling, although constructing one faces several difficulties. Finally, I suggest that the practice of error diagnosis demonstrates that scientists have a tacit-yet-working understanding of their models which has been under-appreciated by some philosophers. (shrink)

Metaphilosophy, Volume 53, Issue 2-3, Page 267-283, April 2022.

Derivational robustness may increase the degree to which various pieces of evidence indirectly confirm a robust result. There are two ways in which this increase may come about. First, if one can show that a result is robust, and that the various individual models used to derive it also have other confirmed results, these other results may indirectly confirm the robust result. Confirmation derives from the fact that data not known to bear on a result are shown to be relevant (...) when it is shown to be robust. Second, robustness may increase the degree to which the robust result is indirectly confirmed if it increases the weight with which existing evidence indirectly confirms it. This may happen when it strengthens the connection between the core and the robust result by showing that auxiliaries are not responsible for the result. (shrink)

The pervasive use of computer simulations in the sciences brings novel epistemological issues discussed in the philosophy of science literature since about a decade. Evolutionary biology strongly relies on such simulations, and in relation to it there exists a research program (Artificial Life) that mainly studies simulations themselves. This paper addresses the specificity of computer simulations in evolutionary biology, in the context (described in Sect. 1) of a set of questions about their scope as explanations, the nature of validation processes (...) and the relation between simulations and true experiments or mathematical models. After making distinctions, especially between a weak use where simulations test hypotheses about the world, and a strong use where they allow one to explore sets of evolutionary dynamics not necessarily extant in our world, I argue in Sect. 2 that (weak) simulations are likely to represent in virtue of the fact that they instantiate specific features of causal processes that may be isomorphic to features of some causal processes in the world, though the latter are always intertwined with a myriad of different processes and hence unlikely to be directly manipulated and studied. I therefore argue that these simulations are merely able to provide candidate explanations for real patterns. Section 3 ends up by placing strong and weak simulations in Levins’ triangle, that conceives of simulations as devices trying to fulfil one or two among three incompatible epistemic values (precision, realism, genericity). (shrink)

Many climate scientists have made claims that may suggest that evidence used in tuning or calibrating a climate model cannot be used to evaluate the model. By contrast, the philosophers Katie Steele and Charlotte Werndl have argued that, at least within the context of Bayesian confirmation theory, tuning is simply an instance of hypothesis testing. In this paper I argue for a weak predictivism and in support of a nuanced reading of climate scientists’ concerns about tuning: there are cases, model-tuning (...) among them, in which predictive successes are more highly confirmatory of a model than accommodation of evidence. (shrink)

Over the last twenty-five years, climate scientists working on the attribution of climate change to humans have developed increasingly sophisticated statistical models in a process that can be understood as a kind of calibration: the gradual changes to the statistical models employed in attribution studies served as iterative revisions to a measurement(-like) procedure motivated primarily by the aim of neutralizing particularly troublesome sources of error or uncertainty. This practice is in keeping with recent work on the evaluation of models more (...) generally that views models as tools for particular tasks: what drives the process is the desire for models that provide more reliable grounds for inference rather than accuracy to the underlying mechanisms of data-generation. (shrink)

I develop an account of productive surprise as an epistemic virtue of scientific investigations which does not turn on psychology alone. On my account, a scientific investigation is potentially productively surprising when results can conflict with epistemic expectations, those expectations pertain to a wide set of subjects. I argue that there are two sources of such surprise in science. One source, often identified with experiments, involves bringing our theoretical ideas in contact with new empirical observations. Another, often identified with simulations, (...) involves articulating and bringing together different parts of our knowledge. Both experiments and simulations, then, can surprise. (shrink)

The paper discusses modelling uncertainties in climate models and how they can be addressed based on physical principles as well as based on how the models perform in light of empirical data. We ar...

An important epistemic issue in climate modelling concerns structural uncertainty: uncertainty about whether the mathematical structure of a model accurately represents its target. How does structural uncertainty affect our knowledge and predictions about the climate? How can we identify sources of structural uncertainty? Can we manage the effect of structural uncertainty on our knowledge claims? These are some of the questions that an epistemology of structural uncertainty faces, and these questions are also important for climate scientists and policymakers. I develop (...) three desiderata for an epistemological account of structural uncertainty. In my view, an account of structural uncertainty should identify sources of structural uncertainty, explain how these sources limit the applicability of a model, and show how the severity of structural uncertainty depends on the questions that can be asked of a model. I argue that analyzing structural uncertainty by paying attention to the details of model building can satisfy these desiderata. I focus on parametrizations, which are representations of important processes occurring at scales that are not resolved by climate models. Parametrizations are often thought to be ad-hoc, but I show that some important parametrizations are theoretically justified by explicit or implicit scale separation assumptions. These assumptions can also be supported empirically. Analyzing these theoretical and empirical justificatory roles of the scale separation assumptions can provide insights into how parametrizations contribute to structural uncertainty. I conclude by sketching how my approach can satisfy the desiderata I set out at the beginning, highlighting its importance for policy-relevant scientific statements about the climate. (shrink)

Many philosophers have argued that a hypothesis is better confirmed by some data if the hypothesis was not specifically designed to fit the data. ‘Prediction’, they argue, is superior to ‘accommodation’. Others deny that there is any epistemic advantage to prediction, and conclude that prediction and accommodation are epistemically on a par. This paper argues that there is a respect in which accommodation is superior to prediction. Specifically, the information that the data was accommodated rather than predicted suggests that the (...) data is less likely to have been manipulated or fabricated, which in turn increases the likelihood that the hypothesis is correct in light of the data. In some cases, this epistemic advantage of accommodation may even outweigh whatever epistemic advantage there might be to prediction, making accommodation epistemically superior to prediction all things considered. (shrink)