The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could (...) then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making. (shrink)

We routinely observe others’ choices and use them to guide our own. Whose choices influence us more, and why? Prior work has focused on the effect of perceived similarity between two individuals, such as the degree of overlap in past choices or explicitly recognizable group affiliations. In the real world, however, any dyadic relationship is part of a more complex social structure involving multiple social groups that are not directly observable. Here we suggest that human learners go beyond dyadic similarities (...) in choice behaviors or explicit group memberships; they infer the structure of social influence by grouping individuals based on choices, and they use these groups to decide whose choices to follow. We propose a computational model that formalizes this idea, and we test the model predictions in a series of behavioral experiments. In Experiment 1, we reproduce a well-established finding that people's choices are more likely to be influenced by someone whose past choices are more similar to their own past choices, as predicted by our model as well as dyadic similarity models. In Experiments 2–5, we test a set of unique predictions of our model by looking at cases where the degree of choice overlap between individuals is equated, but their choices indicate a latent group structure. We then apply our model to prior empirical results on infants’ understanding of others’ preferences, presenting an alternative account of developmental changes. Finally, we discuss how our model relates to classical findings in the social influence literature and the theoretical implications of our model. Taken together, our findings demonstrate that structure learning is a powerful framework for explaining the influence of social information on decision making in a variety of contexts. (shrink)

Flexibility is one of the hallmarks of human problem‐solving. In everyday life, people adapt to changes in common tasks with little to no additional training. Much of the existing work on flexibility in human problem‐solving has focused on how people adapt to tasks in new domains by drawing on solutions from previously learned domains. In real‐world tasks, however, humans must generalize across a wide range of within‐domain variation. In this work we argue that representational abstraction plays an important role in (...) such within‐domain generalization. We then explore the nature of this representational abstraction in realistically complex tasks like video games by demonstrating how the same model‐based planning framework produces distinct generalization behaviors under different classes of task representation. Finally, we compare the behavior of agents with these task representations to humans in a series of novel grid‐based video game tasks. Our results provide evidence for the claim that within‐domain flexibility in humans derives from task representations composed of propositional rules written in terms of objects and relational categories. (shrink)

What place should formal or computational methods occupy in social psychology? We consider this question in historical perspective, survey the current state of the field, introduce the several new contributions to this special issue, and reflect on the future.

In reinforcement learning, a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian model and (...) temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. (shrink)

As modern deep networks become more complex, and get closer to human‐like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations (...) and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances—similar to human zero‐shot reasoning. However, we also note some shortcomings in this generalization behavior—similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human‐like language understanding. (shrink)

Recent progress in artificial intelligence has renewed interest in building systems that learn and think like people. Many advances have come from using deep neural networks trained end-to-end in tasks such as object recognition, video games, and board games, achieving performance that equals or even beats that of humans in some respects. Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways. We review progress in cognitive science suggesting that truly human-like learning and thinking (...) machines will have to reach beyond current engineering trends in both what they learn and how they learn it. Specifically, we argue that these machines should build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems; ground learning in intuitive theories of physics and psychology to support and enrich the knowledge that is learned; and harness compositionality and learning-to-learn to rapidly acquire and generalize knowledge to new tasks and situations. We suggest concrete challenges and promising routes toward these goals that can combine the strengths of recent neural network advances with more structured cognitive models. (shrink)

We were encouraged by the broad enthusiasm for building machines that learn and think in more human-like ways. Many commentators saw our set of key ingredients as helpful, but there was disagreement regarding the origin and structure of those ingredients. Our response covers three main dimensions of this disagreement: nature versus nurture, coherent theories versus theory fragments, and symbolic versus sub-symbolic representations. These dimensions align with classic debates in artificial intelligence and cognitive science, although, rather than embracing these debates, we (...) emphasize ways of moving beyond them. Several commentators saw our set of key ingredients as incomplete and offered a wide range of additions. We agree that these additional ingredients are important in the long run and discuss prospects for incorporating them. Finally, we consider some of the ethical questions raised regarding the research program as a whole. (shrink)