A primary criterion on which models of cognition are evaluated is their ability to fit empirical data. To understand the reason why a model yields a good or poor fit, it is necessary to determine the data‐fitting potential (i.e., flexibility) of the model. In the first part of this article, methods for comparing models and studying their flexibility are reviewed, with a focus on parameter space partitioning (PSP), a general‐purpose method for analyzing and comparing all classes of cognitive models. PSP (...) is then demonstrated in the second part of the article in which two connectionist models of speech perception (TRACE and ARTphone) are compared to learn how design differences affect model flexibility. (shrink)

There is a growing chorus of voices in the scientific community calling for greater openness in the sharing of raw data that lead to a publication. In this commentary, we discuss the merits of sharing, common concerns that are raised, and practical issues that arise in developing a sharing policy. We suggest that the cognitive science community discuss the topic and establish a data-sharing policy.

Experimentation is at the heart of scientific inquiry. In the behavioral and neural sciences, where only a limited number of observations can often be made, it is ideal to design an experiment that leads to the rapid accumulation of information about the phenomenon under study. Adaptive experimentation has the potential to accelerate scientific progress by maximizing inferential gain in such research settings. To date, most adaptive experiments have relied on myopic, one-step-ahead strategies in which the stimulus on each trial is (...) selected to maximize inference on the next trial only. A lingering question in the field has been how much additional benefit would be gained by optimizing beyond the next trial. A range of technical challenges has prevented this important question from being addressed adequately. This study applies dynamic programming, a technique applicable for such full-horizon, “global” optimization, to model-based perceptual threshold estimation, a domain that has been a major beneficiary of adaptive methods. The results provide insight into conditions that will benefit from optimizing beyond the next trial. Implications for the use of adaptive methods in cognitive science are discussed. (shrink)

Norris et al. present a sufficiency case for Merge, but not for autonomy. The simulations make clear that there is little reason to favor Merge over TRACE. The slanted presentation of the empirical evidence gives the illusion that the autonomous position is stronger than it really is.