Analyzing Machine‐Learned Representations: A Natural Language Case Study

Cognitive Science 44 (12):e12925 (2020)
  Copy   BIBTEX


As modern deep networks become more complex, and get closer to human‐like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances—similar to human zero‐shot reasoning. However, we also note some shortcomings in this generalization behavior—similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human‐like language understanding.



    Upload a copy of this work     Papers currently archived: 86,213

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Evolving the Linguistic Mind.Keith Frankish - 2010 - Linguistic and Philosophical Investigations 9:206-214.
An Analytic Tableau System for Natural Logic.Reinhard Muskens - 2010 - In Maria Aloni, H. Bastiaanse, T. De Jager & Katrin Schulz (eds.), Logic, Language and Meaning. Springer. pp. 104-113.
Form and content in semantics.Y. Wilks - 1990 - Synthese 82 (3):329-51.


Added to PP

6 (#1,220,200)

6 months
3 (#337,572)

Historical graph of downloads
How can I increase my downloads?