Do Large Language Models Know What Humans Know?

Cognitive Science 47 (7):e13309 (2023)
  Copy   BIBTEX

Abstract

Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre‐registered analyses, we present a linguistic version of the False Belief Task to both human participants and a large language model, GPT‐3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans nor does it explain the full extent of their behavior—despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 92,100

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Holding Large Language Models to Account.Ryan Miller - 2023 - In Berndt Müller (ed.), Proceedings of the AISB Convention. Society for the Study of Artificial Intelligence and the Simulation of Behaviour. pp. 7-14.
Why not LF for false belief reasoning?Jill G. de Villiers & Peter A. de Villiers - 2002 - Behavioral and Brain Sciences 25 (6):682-683.
You are what you’re for: Essentialist categorization in large language models.Siying Zhang, Selena She, Tobias Gerstenberg & David Rose - forthcoming - Proceedings of the 45Th Annual Conference of the Cognitive Science Society.
The rôle of language in belief revision.Sven Ove Hansson - 2002 - Studia Logica 70 (1):5 - 21.
Belief Change.Sven Ove Hansson - 2012 - In Sven Ove Hansson & Vincent F. Hendricks (eds.), Introduction to Formal Philosophy. Cham: Springer. pp. 401-415.

Analytics

Added to PP
2023-07-06

Downloads
43 (#371,011)

6 months
28 (#109,304)

Historical graph of downloads
How can I increase my downloads?