Commercial success of big data has led to speculation that big-data-like reasoning could partly replace theory-based approaches in science. Big data typically has been applied to ‘small problems’, which are well-structured cases characterized by repeated evaluation of predictions. Here, we show that in climate research, intermediate categories exist between classical domain science and big data, and that big-data elements have also been applied without the possibility of repeated evaluation. Big-data elements can be useful for climate research beyond small problems if (...) combined with more traditional approaches based on domain-specific knowledge. The biggest potential for big-data elements, we argue, lies in socioeconomic climate research. (shrink)

Model evaluation for long-term climate predictions must be done on quantities other than the actual prediction, and a comprehensive uncertainty quantificationUncertainty quantification is impossible. An ad hoc alternative is provided by coordinated model intercomparisonsModel intercomparisons which typically use a “one model one vote” approach. The problem with such an approach is that it treats all models as independent and equally plausible. Reweighting all models of the ensemble for performance and dependence seems like an obvious way to improve on model democracy, (...) yet there are open questions on what constitutes a “good” model, how to define dependency, how to interpret robustnessRobustness, and how to incorporate background knowledgeBackground knowledge. UnderstandingUnderstanding those issues have the potential to increase confidence in model predictions in modeling efforts outside of climate scienceClimate science where similar challenges exist. (shrink)

In recent years, the ability to gather and store information has increased dramatically, and the ability to make use of these increasing volumes of data has improved. This advent of big data has opened up new opportunities for scientific research, including for research on climate change. These changes are associated with a number of interesting philosophical questions. This chapter provides an introduction to these questions. It starts by first clarifying terminological issues concerning “big data” and related terms and by giving (...) an overview of big data elements that can be found in climate research. Second, it discusses data in climate research with a focus on new developments regarding the increase in the volume and complexity of climate data and on how the uncertainty of climate datasets may be assessed. Finally, the chapter addresses the topic of machine learning in climate research and specifically the use of machine learning for the data-driven modeling of climate phenomena. The focus of this discussion is on the representational accuracy of data-driven models and how it might be assessed and what this implies for their use for predictions and for understanding. (shrink)

Model evaluation for long term climate predictions must be done on quantities other than the actual prediction, and a comprehensive uncertainty quantification is impossible. An ad hoc alternative is provided by coordinated model intercomparisons which typically use a “one model one vote” approach. The problem with such an approach is that it treats all models as independent and equally plausible. Reweighting all models of the ensemble for performance and dependence seems like an obvious way to improve on model democracy, yet (...) there are open questions on what constitutes a “good” model, how to define dependency, how to interpret robustness, and how to incorporate background knowledge. Under¬standing those issues has the potential to increase confidence in model predictions in modeling efforts outside of climate science where similar challenges exist. (shrink)

In climate science, observational gridded climate datasets that are based on in situ measurements serve as evidence for scientific claims and they are used to both calibrate and evaluate models. However, datasets only represent selected aspects of the real world, so when they are used for a specific purpose they can be a source of uncertainty. Here, we present a framework for understanding this uncertainty of observational datasets which distinguishes three general sources of uncertainty: (1) uncertainty that arises during the (...) generation of the dataset; (2) uncertainty due to biased samples; and (3) uncertainty that arises due to the choice of abstract properties, such as resolution and metric. Based on this framework, we identify four different types of dataset ensembles — parametric, structural, resampling, and property ensembles—as tools to understand and assess uncertainties arising from the use of datasets for a specific purpose. We advocate for a more systematic generation of dataset ensembles by using these sorts of tools. Finally, we discuss the use of dataset ensembles in climate model evaluation. We argue that a more systematic understanding and assessment of dataset uncertainty is needed to allow for a more reliable uncertainty assessment in the context of model evaluation. The more systematic use of such a framework would be beneficial for both scientific reasoning and scientific policy advice based on climate datasets. (shrink)