A Rational Analysis of the Acquisition of Multisensory Representations

Cognitive Science 36 (2):305-332 (2012)
  Copy   BIBTEX


How do people learn multisensory, or amodal, representations, and what consequences do these representations have for perceptual performance? We address this question by performing a rational analysis of the problem of learning multisensory representations. This analysis makes use of a Bayesian nonparametric model that acquires latent multisensory features that optimally explain the unisensory features arising in individual sensory modalities. The model qualitatively accounts for several important aspects of multisensory perception: (a) it integrates information from multiple sensory sources in such a way that it leads to superior performances in, for example, categorization tasks; (b) its performances suggest that multisensory training leads to better learning than unisensory training, even when testing is conducted in unisensory conditions; (c) its multisensory representations are modality invariant; and (d) it predicts ‘‘missing” sensory representations in modalities when the input to those modalities is absent. Our rational analysis indicates that all of these aspects emerge as part of the optimal solution to the problem of learning to represent complex multisensory environments



    Upload a copy of this work     Papers currently archived: 92,261

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Modality, quo vadis?K. Sathian - 2004 - Behavioral and Brain Sciences 27 (3):413-414.
The unity of haptic touch.Matthew Fulkerson - 2011 - Philosophical Psychology 24 (4):493 - 516.
Probability rather than logic as the basis of perception.Thomas J. Anastasio - 2003 - Behavioral and Brain Sciences 26 (3):283-284.
The multisensory perception of flavor.Malika Auvray & Charles Spence - 2008 - Consciousness and Cognition 17 (3):1016-1031.
Active Perception and the Representation of Space.Mohan Matthen - 2014 - In Dustin Stokes, Mohan Matthen & Stephen Biggs (eds.), Perception and Its Modalities. Oxford University Press. pp. 44-72.


Added to PP

110 (#161,675)

6 months
6 (#530,265)

Historical graph of downloads
How can I increase my downloads?

References found in this work

Vision.David Marr - 1982 - W. H. Freeman.
Judgement under Uncertainty: Heuristics and Biases.Daniel Kahneman, Paul Slovic & Amos Tversky - 1985 - British Journal for the Philosophy of Science 36 (3):331-340.

View all 10 references / Add more references