Scientific Evidence, Big Data and the Curse of Dimensionality

Abstract

The curse of dimensionality is one of the most prominent challenge that data scientists face when trying to make valuable inferences. It is an epistemic problem that hits particularly hard in "Big Data" research contexts, where the volume of the data set is particularly large. The way in which we tackle with this problem sheds light on the notion of scientific evidence. Yet, it is virtually absent from the current philosophical literature. In this paper, I aim to broaden the focus of that literature by showing that the dimensions in which the data are embedded are an integral part of scientific evidence. This is an aspect of scientific evidence that is often hidden behind the traditional observation/theory dichotomy that we often encounter in philosophy of science. Dimensions are abstract objects. They are not observables like tables and chairs, nor are they entries in a data set. Ultimately, I aim to show that empirical adequacy is not merely a matter of finding the model that best fit the relevant observational data. It is also a matter of finding the model that best fit the relevant observational data inside the relevant dimensions.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 91,532

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

  • Only published works are available at libraries.

Similar books and articles

What Counts as Scientific Data? A Relational Framework.Sabina Leonelli - 2015 - Philosophy of Science 82 (5):810-821.
The epistemology of evidence in cognitive neuroscience.William P. Bechtel - forthcoming - In R. Skipper Jr, C. Allen, R. A. Ankeny, C. F. Craver, L. Darden, G. Mikkelson & and R. Richardson (eds.), Philosophy and the Life Sciences: A Reader. MIT Press.
Theory-ladenness of evidence: a case study from history of chemistry.Prajit K. Basu - 2003 - Studies in History and Philosophy of Science Part A 34 (2):351-368.
Theory-ladenness of evidence: A case study from history of chemistry.K. P. - 2003 - Studies in History and Philosophy of Science Part A 34 (2):351-368.
Data models and the acquisition and manipulation of data.Todd Harris - 2003 - Philosophy of Science 70 (5):1508-1517.
Issues in Data Management.Sharon S. Krag - 2010 - Science and Engineering Ethics 16 (4):743-748.
Some doubts about scientific data.Gordon N. Pinkham - 1975 - Philosophy of Science 42 (3):260-269.
Scientists' Responses to Anomalous Data: Evidence from Psychology, History, and Philosophy of Science.William F. Brewer & Clark A. Chinn - 1994 - PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association 1994:304 - 313.
Data, Evidence, and Explanatory Power.Pascal Ströing - 2018 - Philosophy of Science 85 (3):422-441.
Scientific perspectivism: A philosopher of science's response to the challenge of big data biology.Werner Callebaut - 2012 - Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences 43 (1):69-80.

Analytics

Added to PP
2018-07-18

Downloads
31 (#511,808)

6 months
8 (#351,349)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Citations of this work

No citations found.

Add more citations

References found in this work

Theory and observation in science.Jim Bogen - 2009 - Stanford Encyclopedia of Philosophy.
Saving Unobservable Phenomena.Michela Massimi - 2007 - British Journal for the Philosophy of Science 58 (2):235-262.

Add more references