This article reviews current methods for evaluating models in the cognitive sciences, including theoretically based approaches, such as Bayes factors and minimum description length measures; simulation approaches, including model mimicry evaluations; and practical approaches, such as validation and generalization measures. This article argues that, although often useful in specific settings, most of these approaches are limited in their ability to give a general assessment of models. This article argues that hierarchical methods, generally, and hierarchical Bayesian methods, specifically, can provide a (...) more thorough evaluation of models in the cognitive sciences. This article presents two worked examples of hierarchical Bayesian analyses to demonstrate how the approach addresses key questions of descriptive adequacy, parameter interference, prediction, and generalization in principled and coherent ways. (shrink)

If we are eventually to understand the capability of higher organisms for perceptual recognition, generalization, recall, and thinking, we must first have answers to three fundamental questions: 1. How is information about the physical world sensed, or detected, by the biological system? 2. In what form is information stored, or remembered? 3. How does information contained in storage, or in memory, influence recognition and behavior? The first of these questions is in the.

A primary criterion on which models of cognition are evaluated is their ability to fit empirical data. To understand the reason why a model yields a good or poor fit, it is necessary to determine the data‐fitting potential (i.e., flexibility) of the model. In the first part of this article, methods for comparing models and studying their flexibility are reviewed, with a focus on parameter space partitioning (PSP), a general‐purpose method for analyzing and comparing all classes of cognitive models. PSP (...) is then demonstrated in the second part of the article in which two connectionist models of speech perception (TRACE and ARTphone) are compared to learn how design differences affect model flexibility. (shrink)

This article investigates the potential of fMRI to test assumptions about different components in models of complex cognitive tasks. If the components of a model can be associated with specific brain regions, one can make predictions for the temporal course of the BOLD response in these regions. An event‐locked procedure is described for dealing with temporal variability and bringing model runs and individual data trials into alignment. Statistical methods for testing the model are described that deal with the scan‐to‐scan correlations (...) in the errors of measurement of the BOLD signal. This approach is illustrated using a “sacrificial” ACT‐R model that involves mapping 6 modules onto 6 brain regions in an experiment from Ravizza, Anderson, and Carter (in press) concerned with equation solving. The model's visual encoding predicted the BOLD response in the fusiform gyrus, its controlled retrieval predicted the BOLD response in the lateral inferior prefrontal cortex, and its subgoal setting predicted the BOLD response in the anterior cingulate cortex. On the other hand, its motor programming failed to predict anticipatory activation in the motor cortex, its representational changes failed to predicted the pattern of activity in the posterior parietal cortex, and its procedural component failed to predict an initial spike in caudate. The results illustrate the power of such data to direct the development of a theory of complex problem solving, both at the level of a specific task model as well as at the level of the cognitive architecture. (shrink)

This is the first textbook on pattern recognition to present the Bayesian viewpoint. The book presents approximate inference algorithms that permit fast approximate answers in situations where exact answers are not feasible. It uses graphical models to describe probability distributions when no other books apply graphical models to machine learning. No previous knowledge of pattern recognition or machine learning concepts is assumed. Familiarity with multivariate calculus and basic linear algebra is required, and some experience in the use of probabilities would (...) be helpful though not essential as the book includes a self-contained introduction to basic probability theory. (shrink)

In this book, Newell makes the case for unified theories by setting forth a candidate.