Lost in a random forest: Using Big Data to study rare events

Big Data and Society 2 (2) (2015)
  Copy   BIBTEX

Abstract

Sudden, broad-scale shifts in public opinion about social problems are relatively rare. Until recently, social scientists were forced to conduct post-hoc case studies of such unusual events that ignore the broader universe of possible shifts in public opinion that do not materialize. The vast amount of data that has recently become available via social media sites such as Facebook and Twitter—as well as the mass-digitization of qualitative archives provide an unprecedented opportunity for scholars to avoid such selection on the dependent variable. Yet the sheer scale of these new data creates a new set of methodological challenges. Conventional linear models, for example, minimize the influence of rare events as “outliers”—especially within analyses of large samples. While more advanced regression models exist to analyze outliers, they suffer from an even more daunting challenge: equifinality, or the likelihood that rare events may occur via different causal pathways. I discuss a variety of possible solutions to these problems—including recent advances in fuzzy set theory and machine learning—but ultimately advocate an ecumenical approach that combines multiple techniques in iterative fashion.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 91,349

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Structure from interaction events.Wouter de Nooy - 2015 - Big Data and Society 2 (2).
Quantum Field Theory of Black-Swan Events.H. Kleinert - 2014 - Foundations of Physics 44 (5):546-556.
Rare events research.Will Lowe - 2004 - In Kimberly Kempf-Leonard (ed.), Encyclopedia of Social Measurement. Elsevier. pp. 293--297.
Discussion on Media Report on Group Events.Fanbin Zeng - 2012 - Asian Culture and History 4 (1):p54.
Kim on Events.Simone Gozzano - 2015 - Metaphysica 16 (2).
Could anyone justifiably believe epiphenomenalism?Richard Swinburne - 2011 - Journal of Consciousness Studies 18 (3-4):196--216.
A Bayesian Baseline for Belief in Uncommon Events.Vesa Palonen - 2017 - European Journal for Philosophy of Religion 9 (3):159-175.
Mental models in data interpretation.Clark A. Chinn & William F. Brewer - 1996 - Philosophy of Science 63 (3):219.

Analytics

Added to PP
2020-11-24

Downloads
5 (#1,505,296)

6 months
3 (#1,023,809)

Historical graph of downloads
How can I increase my downloads?