Seeing and speaking: How verbal 'description length' encodes visual complexity

Journal of Experimental Psychology: General (1):82-96 (2021)
  Copy   BIBTEX

Abstract

What is the relationship between complexity in the world and complexity in the mind? Intuitively, increasingly complex objects and events should give rise to increasingly complex mental representations (or perhaps a plateau in complexity after a certain point). However, a counterintuitive possibility with roots in information theory is an inverted U-shaped relationship between the “objective” complexity of some stimulus and the complexity of its mental representation, because excessively complex patterns might be characterized by surprisingly short computational descriptions (e.g., if they are represented as having been generated “randomly”). Here, we demonstrate that this is the case, using a novel approach that takes the notion of “description” literally. Subjects saw static and dynamic visual stimuli whose objective complexity could be carefully manipulated, and they described these stimuli in their own words by giving freeform spoken descriptions of them. Across three experiments totaling over 10,000 speech clips, spoken descriptions of shapes (Experiment 1), dot-arrays (Experiment 2), and dynamic motion-paths (Experiment 3) revealed a striking quadratic relationship between the raw complexity of these stimuli and the length of their spoken descriptions. In other words, the simplest and most complex stimuli received the shortest descriptions, while those stimuli with a “medium” degree of complexity received the longest descriptions. Follow-up analyses explored the particular words used by subjects, allowing us to further explore how such stimuli were represented. We suggest that the mind engages in a kind of lossy compression for overly complex stimuli, and we discuss the utility of such freeform responses for exploring foundational questions about mental representation.

Similar books and articles

Curious objects: How visual complexity guides attention and engagement.Zekun Sun & Chaz Firestone - 2021 - Cognitive Science: A Multidisciplinary Journal 45 (4):e12933.
Abstract Complexity Definition.Mariusz Stanowski - 2011 - Complicity: An International Journal of Complexity and Education (2).
Kolmogorov Complexity and Noncomputability.George Davie - 2002 - Mathematical Logic Quarterly 48 (4):574-581.
You never get a second chance to make a first impression: The effect of visual complexity on intention to use websites.Rik Crutzen, Linda deKruif & Nanne K. deVries - 2012 - Interaction Studiesinteraction Studies Social Behaviour and Communication in Biological and Artificial Systems 13 (3):469-477.
Kolmogorov Complexity and Noncomputability.George Davie - 2002 - Mathematical Logic Quarterly 48 (5):574-580.
The Genesis of Complexity.Ralph H. Abraham - 2011 - World Futures 67 (4-5):380 - 394.

Analytics

Added to PP
2021-07-30

Downloads
618 (#26,550)

6 months
149 (#19,795)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Chaz Firestone
Johns Hopkins University

Citations of this work

Compositionality in visual perception.Alon Hafri, E. J. Green & Chaz Firestone - 2023 - Behavioral and Brain Sciences 46:e277.

Add more citations

References found in this work

Language and Mind.Noam Chomsky - 1968 - Cambridge University Press.
A Mathematical Theory of Communication.Claude Elwood Shannon - 1948 - Bell System Technical Journal 27 (April 1924):379–423.

View all 21 references / Add more references