A central controversy in cognitive science concerns the roles of rules versus similarity. To gain some leverage on this problem, we propose that rule‐ versus similarity‐based processes can be characterized as extremes in a multidimensional space that is composed of at least two dimensions: the number of features (Pothos, 2005) and the physical presence of features. The transition of similarity‐ to rule‐based processing is conceptualized as a transition in this space. To illustrate this, we show how a neural network model (...) uses input features (and in this sense produces similarity‐based responses) when it has a low learning rate or in the early phases of training, but it switches to using self‐generated, more abstract features (and in this sense produces rule‐based responses) when it has a higher learning rate or is in the later phases of training. Relations with categorization and the psychology of learning are pointed out. (shrink)

Human participants and recurrent (“connectionist”) neural networks were both trained on a categorization system abstractly similar to natural language systems involving irregular (“strong”) classes and a default class. Both the humans and the networks exhibited staged learning and a generalization pattern reminiscent of the Elsewhere Condition (Kiparsky, 1973). Previous connectionist accounts of related phenomena have often been vague about the nature of the networks’ encoding systems. We analyzed our network using dynamical systems theory, revealing topological and geometric properties that can (...) be directly compared with the mechanisms of non-connectionist, rule-based accounts. The results reveal that the networks “contain” structures related to mechanisms posited by rule-based models, partly vindicating the insights of these models. On the other hand, they support the one mechanism (OM), as opposed to the more than one mechanism (MOM), view of symbolic abstraction by showing how the appearance of MOM behavior can arise emergently from one underlying set of principles. The key new contribution of this study is to show that dynamical systems theory can allow us to explicitly characterize the relationship between the two perspectives in implemented models. (shrink)

Phonemes have variant pronunciations depending on context. For instance, in American English, the [t] in pat [pæt] and the [d] in pad [pæd] are both realized with a tap [ɾ] when the –ing suffix is attached, [pæɾɪŋ]. We show that despite greater distributional and acoustic support for the [t]-tap alternation, 12-month-olds successfully relate taps to stems with a perceptually-similar final [d], not the dissimilar final-[t]. Thus, distributional learning of phonological alternations is constrained by infants' preference for the alternation of perceptually-similar (...) segments. Further, the ability to relate variant surface forms emerges between 8- and 12-months. Our findings of biased learning provide further empirical support for a role for perceptual similarity in the acquisition of linguistically-relevant categories. We discuss the implications of our findings for phonological theory, language acquisition and models of the mental lexicon. (shrink)

Rumelhart and McClelland's chapter about learning the past tense created a degree of controversy extraordinary even in the adversarial culture of modern science. It also stimulated a vast amount of research that advanced the understanding of the past tense, inflectional morphology in English and other languages, the nature of linguistic representations, relations between language and other phenomena such as reading and object recognition, the properties of artificial neural networks, and other topics. We examine the impact of the Rumelhart and McClelland (...) model with the benefit of 25 years of hindsight. It is not clear who “won” the debate. It is clear, however, that the core ideas that the model instantiated have been assimilated into many areas in the study of language, changing the focus of research from abstract characterizations of linguistic competence to an emphasis on the role of the statistical structure of language in acquisition and processing. (shrink)

The input to phonological reasoning are alternations, that is, variations in the pronunciation of related words, such as in electri[k] ‐ electri[s]‐ity. But phonologists cannot agree what counts as a relevant alternation: the issue is highly contentious despite a research record of over 50 years. We believe that the experimental setup presented may contribute to this debate based on a kind of evidence that was not brought to bear to date. Our experiment was thus designed to distinguish between alternations where (...) phonological computation plays no role, and those where it has contributed to language production. The design manipulates two factors that to date have not been considered in experimental studies of language production: linguistic complexity and alternation specificity. The former is understood as extra processing demands issued by two types of linguistic activity, morphosyntactic and phonological. Our results show that reaction time latencies are longer when participants are burdened with both morphosyntactic and phonological tasks than when they carry out just a morphosyntactic task, and they are still shorter in absence of both types of demands. These results allowed us to address alternation specificity, that is, the fact that different alternations (within a language or across languages) may be driven by different production routines (an idea that is consensual among linguists but underdeveloped in the psycholinguistic literature). Our study shows that four different alternations in French produce alternation‐specific signatures in reaction time latencies. These findings may thus redound to the advantage of psycholinguistic studies by identifying two new factors, as well as produce results that speak to the linguistic quarrels. (shrink)

Mental time travel and language enable us to go back and forth in time and to organize and express our personal experiences through time reference. People with Alzheimer’s disease have both mental time travel and time reference impairments, which can greatly impact their daily communication. Currently, little is known about the potential relationship between time conceptualization and time reference difficulties in this disease. A systematic review of the literature was performed to determine if this link had already been investigated. Only (...) three articles integrated both time conceptualization and time reference measures. However, the link between the two was not systematically analyzed and interpreted. This review highlights the lack of research addressing the question of the influence of time conceptualization impairments in Alzheimer’s disease on other cognitive domains, and especially language. (shrink)

Many generativist accounts argue for very early knowledge of inflection on the basis of very low rates of person/number marking errors in young children's speech. However, studies of Spanish and Brazilian Portuguese have revealed that these low overall error rates actually hide important differences across the verb paradigm. The present study investigated children's production of person/number marked verbs by eliciting present tense verb forms from 82 native Finnish-speaking children aged 2;2–4;8 years. Four main findings were observed: Rates of person/number marking (...) errors were higher in low-frequency person/number contexts, even excluding children who showed no evidence of having learned the relevant morpheme, most errors involved the use of higher frequency forms in lower frequency person/number contexts, error rates were predicted not only by the frequency of person/number contexts but also by the frequency of individual “ready-inflected” lexical target forms, and for low-frequency verbs, lower error rates were observed for verbs with high phonological neighborhood density. It is concluded that any successful account of the development of verb inflection will need to incorporate both rote-storage and retrieval of individual inflected forms and phonological analogy across them. (shrink)

This paper introduces a special issue of Cognitive Science initiated on the 25th anniversary of the publication of Parallel Distributed Processing (PDP), a two-volume work that introduced the use of neural network models as vehicles for understanding cognition. The collection surveys the core commitments of the PDP framework, the key issues the framework has addressed, and the debates the framework has spawned, and presents viewpoints on the current status of these issues. The articles focus on both historical roots and contemporary (...) developments in learning, optimality theory, perception, memory, language, conceptual knowledge, cognitive control, and consciousness. Here we consider the approach more generally, reviewing the original motivations, the resulting framework, and the central tenets of the underlying theory. We then evaluate the impact of PDP both on the field at large and within specific subdomains of cognitive science and consider the current role of PDP models within the broader landscape of contemporary theoretical frameworks in cognitive science. Looking to the future, we consider the implications for cognitive science of the recent success of machine learning systems called “deep networks”—systems that build on key ideas presented in the PDP volumes. (shrink)

We distinguish between literal and metaphorical applications of Bayesian models. When intended literally, an isomorphism exists between the elements of representation assumed by the rational analysis and the mechanism that implements the computation. Thus, observation of the implementation can externally validate assumptions underlying the rational analysis. In other applications, no such isomorphism exists, so it is not clear how the assumptions that allow a Bayesian model to fit data can be independently validated.

In a series of studies children show increasing mastery of irregular plural forms (such as mice) simply by producing erroneous over‐regularized versions of them (such as mouses). We explain this phenomenon in terms of successive approximation in imitation: Children over‐regularize early in acquisition because the representations of frequent, regular plural forms develop more quickly, such that at the earliest stages of production they interfere with children's attempts to imitatively reproduce irregular forms they have heard in the input. As the strength (...) of the representations that determine children's productions settle asymptotically, the early advantage for the frequent regular forms is negated, and children's attempts to imitate the irregular forms they have observed become more likely to succeed (a process that produces the classic U‐shape in children's acquisition of plural inflection). These data show that children can acquire correct linguistic behavior without feedback in a situation where, as a result of philosophical and linguistic analyses, it has often been argued that it is logically impossible for them to do so. (shrink)

What is the interaction between storage and computation in language processing? What is the psychological status of grammatical rules? What are the relative strengths of connectionist and symbolic models of cognition? How are the components of language implemented in the brain? The English past tense has served as an arena for debates on these issues. We defend the theory that irregular past-tense forms are stored in the lexicon, a division of declarative memory, whereas regular forms can be computed by a (...) concatenation rule, which requires the procedural system. Irregulars have the psychological, linguistic and neuropsychological signatures of lexical memory, whereas regulars often have the signatures of grammatical processing. Furthermore, because regular inflection is rule-driven, speakers can apply it whenever memory fails. (shrink)

According to some philosophers, computational explanation is proprietary
to psychology—it does not belong in neuroscience. But neuroscientists routinely offer computational explanations of cognitive phenomena. In fact, computational explanation was initially imported from computability theory into the science of mind by neuroscientists, who justified this move on neurophysiological grounds. Establishing the legitimacy and importance of computational explanation in neuroscience is one thing; shedding light on it is another. I raise some philosophical questions pertaining to computational explanation and outline some promising answers that (...) are being developed by a number of authors. (shrink)

Newport addresses a fundamental question in language learning: When, why, and how do learners come to form rules, given linguistic input that varies probabilistically? She presents several case studies that confirm and extend a long‐standing theme of her work: that young learners tend to form rules from variable input, whereas adult learners store and use its statistical probabilities. Thus, child and adult learners use quite different kinds of computations when learning language; the consequence is that operating on the very same (...) input to learning results in quite different outcomes of learning. (shrink)

Inflectional morphology has been taken as a paradigmatic example of rule-governed grammatical knowledge (Pinker, 1999). The plausibility of this claim may be related to the fact that it is mainly based on studies of English, which has a very simple inflectional system. We examined the representation of inflectional morphology in Serbian, which encodes number, gender, and case for nouns. Linguists standardly characterize this system as a complex set of rules, with disagreements about their exact form. We present analyses of a (...) large corpus of nouns which showed that, as in English, Serbian inflectional morphology is quasiregular: It exhibits numerous partial regularities creating neighborhoods that vary in size and consistency. We then asked whether a simple connectionist network could encode this statistical information in a manner that also supported generalization. A network trained on 3,244 Serbian nouns learned to produce correctly inflected phonological forms from a specification of a word's lemma, gender, number, and case, and generalized to untrained cases. The model's performance was sensitive to variables that also influence human performance, including surface and lemma frequency. It was also influenced by inflectional neighborhood size, a novel measure of the consistency of meaning to form mapping. A word-naming experiment with native Serbian speakers showed that this measure also affects human performance. The results suggest that, as in English, generating correctly inflected forms involves satisfying a small number of simultaneous probabilistic constraints relating form and meaning. Thus, common computational mechanisms may govern the representation and use of inflectional information across typologically diverse languages. (shrink)

I consider the role of cognitive modeling in cognitive science. Modeling, and the computers that enable it, are central to the field, but the role of modeling is often misunderstood. Models are not intended to capture fully the processes they attempt to elucidate. Rather, they are explorations of ideas about the nature of cognitive processes. In these explorations, simplification is essential—through simplification, the implications of the central ideas become more transparent. This is not to say that simplification has no downsides; (...) it does, and these are discussed. I then consider several contemporary frameworks for cognitive modeling, stressing the idea that each framework is useful in its own particular ways. Increases in computer power (by a factor of about 4 million) since 1958 have enabled new modeling paradigms to emerge, but these also depend on new ways of thinking. Will new paradigms emerge again with the next 1,000‐fold increase? (shrink)

The study of human intelligence was once dominated by symbolic approaches, but over the last 30 years an alternative approach has arisen. Symbols and processes that operate on them are often seen today as approximate characterizations of the emergent consequences of sub- or nonsymbolic processes, and a wide range of constructs in cognitive science can be understood as emergents. These include representational constructs (units, structures, rules), architectural constructs (central executive, declarative memory), and developmental processes and outcomes (stages, sensitive periods, neurocognitive (...) modules, developmental disorders). The greatest achievements of human cognition may be largely emergent phenomena. It remains a challenge for the future to learn more about how these greatest achievements arise and to emulate them in artificial systems. (shrink)

We share with Anderson & Lebiere (A&L) (and with Newell before them) the goal of developing a domain-general framework for modeling cognition, and we take seriously the issue of evaluation criteria. We advocate a more focused approach than the one reflected in Newell's criteria, based on analysis of failures as well as successes of models brought into close contact with experimental data. A&L attribute the shortcomings of our parallel-distributed processing framework to a failure to acknowledge a symbolic level of thought. (...) Our framework does acknowledge a symbolic level, contrary to their claim. What we deny is that the symbolic level is the level at which the principles of cognitive processing should be formulated. Models cast at a symbolic level are sometimes useful as high-level approximations of the underlying mechanisms of thought. The adequacy of this approximation will continue to increase as symbolic modelers continue to incorporate principles of parallel distributed processing. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this (...) manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this (...) manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

Since several years ago, there is a debate about morphological processing. We have two different perspectives, one simple and another dual. The key difference is that latter accepts the existence of specific rules to morphological processing, while that simple one postulates that there is not rules because all complex morphologically words are stored in mental lexicon. In this paper, our goal is to analyze the neologisms created by four sensorial aphasic speakers to determine if constructions follow specific morphological rules or (...) not. First, we collected data through interviews; second, we identified neologisms using syntactic criteria and, then, we analyze its composition according to grammatical rules of Spanish. Our results suggest that sensorial aphasic speakers do respect the morphological rules when they create this sort of neologisms. Finally, evidence show that exist a dual-route to morphological processing to verbs in Spanish. (shrink)

Generative grammarians typically advocate for a rationalist understanding of language acquisition, according to which the structure of a developed language faculty reflects innate guidance rather than environmental influence. This proposal is developed in developmental linguistics by triggering models of language acquisition. Opposing this tradition, various theorists have advocated for empiricist views of language acquisition, according to which the structure of a developed linguistic competence reflects the linguistic environment in which this competence developed. On this picture, linguistic development is accounted for (...) by general statistical learning mechanisms. In this article I shall precisify the debate, provide a clearer picture of what is at stake, and show why an intermediate picture is needed. (shrink)

Computational models will play an important role in our understanding of human higher‐order cognition. How can a model's contribution to this goal be evaluated? This article argues that three important aspects of a model of higher‐order cognition to evaluate are (a) its ability to reason, solve problems, converse, and learn as well as people do; (b) the breadth of situations in which it can do so; and (c) the parsimony of the mechanisms it posits. This article argues that fits of (...) models to quantitative experimental data, although valuable for other reasons, do not address these criteria. Further, using analogies with other sciences, the history of cognitive science, and examples from modern‐day research programs, this article identifies five activities that have been demonstrated to play an important role in our understanding of human higher‐order cognition. These include modeling within a cognitive architecture, conducting artificial intelligence research, measuring and expanding a model's ability, finding mappings between the structure of different domains, and attempting to explain multiple phenomena within a single model. (shrink)

A U-shaped curve in a cognitive-developmental trajectory refers to a three-step process: good performance followed by bad performance followed by good performance once again. U-shaped curves have been observed in a wide variety of cognitive-developmental and learning contexts. U-shaped learning seems to contradict the idea that learning is a monotonic, cumulative process and thus constitutes a challenge for competing theories of cognitive development and learning. U-shaped behavior in language learning (in particular in learning English past tense) has become a central (...) topic in the Cognitive Science debate about learning models. Antagonist models (e.g., connectionism versus nativism) are often judged on their ability of modeling or accounting for U-shaped behavior. The prior literature is mostly occupied with explaining how U-shaped behavior occurs. Instead, we are interested in the necessity of this kind of apparently inefficient strategy. We present and discuss a body of results in the abstract mathematical setting of (extensions of) Gold-style computational learning theory addressing a mathematically precise version of the following question: Are there learning tasks that require U-shaped behavior? All notions considered are learning in the limit from positive data. We present results about the necessity of U-shaped learning in classical models of learning as well as in models with bounds on the memory of the learner. The pattern emerges that, for parameterized, cognitively relevant learning criteria, beyond very few initial parameter values, U-shapes are necessary for full learning power! We discuss the possible relevance of the above results for the Cognitive Science debate about learning models as well as directions for future research. (shrink)

Productivity—the hallmark of linguistic competence—is typically attributed to algebraic rules that support broad generalizations. Past research on spoken language has documented such generalizations in both adults and infants. But whether algebraic rules form part of the linguistic competence of signers remains unknown. To address this question, here we gauge the generalization afforded by American Sign Language (ASL). As a case study, we examine reduplication (X→XX)—a rule that, inter alia, generates ASL nouns from verbs. If signers encode this rule, then they (...) should freely extend it to novel syllables, including ones with features that are unattested in ASL. And since reduplicated disyllables are preferred in ASL, such rule should favor novel reduplicated signs. Novel reduplicated signs should thus be preferred to nonreduplicative controls (in rating), and consequently, such stimuli should also be harder to classify as nonsigns (in the lexical decision task). The results of four experiments support this prediction. These findings suggest that the phonological knowledge of signers includes powerful algebraic rules. The convergence between these conclusions and previous evidence for phonological rules in spoken language suggests that the architecture of the phonological mind is partly amodal. (shrink)