Probabilistic methods are providing new explanatory approaches to fundamental cognitive science questions of how humans structure, process and acquire language. This review examines probabilistic models defined over traditional symbolic structures. Language comprehension and production involve probabilistic inference in such models; and acquisition involves choosing the best model, given innate constraints and linguistic and other input. Probabilistic models can account for the learning and processing of language, while maintaining the sophistication of symbolic models. A recent burgeoning (...) of theoretical developments and online corpus creation has enabled large models to be tested, revealing probabilistic constraints in processing, undermining acquisition arguments based on a perceived poverty of the stimulus, and suggesting fruitful links with probabilistic theories of categorization and ambiguity resolution in perception. (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this (...) manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

Natural language is full of patterns that appear to fit with general linguistic rules but are ungrammatical. There has been much debate over how children acquire these “linguistic restrictions,” and whether innate language knowledge is needed. Recently, it has been shown that restrictions in language can be learned asymptotically via probabilistic inference using the minimum description length (MDL) principle. Here, we extend the MDL approach to give a simple and practical methodology for estimating how much linguistic data are required (...) to learn a particular linguistic restriction. Our method provides a new research tool, allowing arguments about natural language learnability to be made explicit and quantified for the first time. We apply this method to a range of classic puzzles in language acquisition. We find some linguistic rules appear easily statistically learnable from language experience only, whereas others appear to require additional learning mechanisms (e.g., additional cues or innate constraints). (shrink)

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural‐language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this (...) manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

Entrenchment and schematisation are the two most important cognitive processes in language acquisition. In this article, the role of the two processes, operationalised by token and type frequency, in the production of overgeneralised verb forms in Croatian preschool children is investigated using a parental questionnaire and computational simulation of language acquisition. The participants of the questionnaire were parents of children aged 3;0–5;11 years (n = 174). The results showed that parents of most children (93 %) reported the parallel (...) use of both adult-like and overgeneralised verb forms, suggesting that Croatian-speaking preschool children have not yet fully acquired the verbal system. The likelihood of overgeneralised forms being reported decreases with the age of the children and verb type frequency. The results of the computational simulation show that patterns with a higher type frequency also show a greater preference for the correct form, while lexical items show both learning and unlearning tendencies during the process. (shrink)

Words are the essence of communication: They are the building blocks of any language. Learning the meaning of words is thus one of the most important aspects of language acquisition: Children must first learn words before they can combine them into complex utterances. Many theories have been developed to explain the impressive efficiency of young children in acquiring the vocabulary of their language, as well as the developmental patterns observed in the course of lexical acquisition. A major source (...) of disagreement among the different theories is whether children are equipped with special mechanisms and biases for word learning, or their general cognitive abilities are adequate for the task. We present a novel computational model of early word learning to shed light on the mechanisms that might be at work in this process. The model learns word meanings as probabilistic associations between words and semantic elements, using an incremental and probabilistic learning mechanism, and drawing only on general cognitive abilities. The results presented here demonstrate that much about word meanings can be learned from naturally occurring child-directed utterances (paired with meaning representations), without using any special biases or constraints, and without any explicit developmental changes in the underlying learning mechanism. Furthermore, our model provides explanations for the occasionally contradictory child experimental data, and offers predictions for the behavior of young word learners in novel situations. (shrink)

We propose a new model for forming and revising beliefs about unknown probabilities. To go beyond what is known with certainty and represent the agent’s beliefs about probability, we consider a plausibility map, associating to each possible distribution a plausibility ranking. Beliefs are defined as in Belief Revision Theory, in terms of truth in the most plausible worlds. We consider two forms of conditioning or belief update, corresponding to the acquisition of two types of information: learning observable evidence obtained (...) by repeated sampling from the unknown distribution; and learning higher-order information about the distribution. The first changes only the plausibility map, but leaves the given set of possible distributions essentially unchanged; the second rules out some distributions, thus shrinking the set of possibilities, without changing their plausibility ordering.. We look at stability of beliefs under either of these types of learning, defining two related notions, as well as a measure of the verisimilitude of a given plausibility model. We prove a number of convergence results, showing how our agent’s beliefs track the true probability after repeated sampling, and how she eventually gains in a sense knowledge of that true probability. Finally, we sketch the contours of a dynamic doxastic logic for statistical learning. (shrink)

Theories of relational concept acquisition based on structured intersection discovery predict that relational concepts with a probabilistic structure ought to be extremely difficult to learn. We report four experiments testing this prediction by investigating conditions hypothesized to facilitate the learning of such categories. Experiment 1 showed that changing the task from a category-learning task to choosing the “winning” object in each stimulus greatly facilitated participants' ability to learn probabilistic relational categories. Experiments 2 and 3 further investigated the (...) mechanisms underlying this “who's winning” effect. Experiment 4 replicated and generalized the “who's winning” effect with more natural stimuli. Together, our findings suggest that people learn relational concepts by a process of intersection discovery akin to schema induction, and that any task that encourages people to discover a higher order relation that remains invariant over members of a category will facilitate the learning of putatively probabilistic relational concepts. (shrink)

L’objectif de ce travail de recherche est d’étudier la distinction formelle entre les locutions verbales et les constructions à verbe support telles qu’elles sont présentées et décrites par les chercheurs travaillant dans le cadre théorique du lexique-grammaire. Dans ce but, nous avons conçu quatre tâches que nous avons proposées à nos deux groupes de participants : des locuteurs natifs du français et des apprenants du FL2. Nous avons testé plusieurs aspects de la maîtrise des constructions verbales complexes en français par (...) nos participants en mettant en lumière les différences dans les tâches de perception ainsi que dans les tâches de production des structures ciblées. Cette étude démontre que, bien que la perception des structures verbales complexes par les apprenants du FL2 soit assez semblable à celle des locuteurs natifs de cette langue, leur production représente un aspect assez difficile pour les premiers. (shrink)

Participants were asked to draw inferences about correlation from single x,y observations. In Experiment 1 statistically sophisticated participants were given the univariate characteristics of distributions of x and y and asked to infer whether a single x, y observation came from a correlated or an uncorrelated population. In Experiment 2, students with a variety of statistical backgrounds assigned posterior probabilities to five possible populations based on single x, y observations, again given knowledge of the univariate statistics. In Experiment 3, statistically (...) naïve participants were given a problem analogous to that given in Experiment 1, framed verbally. Experiment 4 replicated Experiment 3 but added an “impossible to determine” response option. Models that rely on computing sample correlations make no predictions about these investigations. From a Bayesian perspective, participants’ inferences in all four experiments tended to make probabilistically valid inferences as long as the single datum was directional. The results are discussed in light of the Brunswikian notion of vicarious functioning. (shrink)

The concepts of the innateness, universality, species-specificity, and autonomy of the human language capacity have had an extreme impact on the psycholinguistic debate for over thirty years. These concepts are evaluated from several neurobiological perspectives, with an emphasis on the emergence of language and its decay due to brain lesion and progressive brain disease.Evidence of perceptuomotor homologies and preadaptations for human language in nonhuman primates suggests a gradual emergence of language during hominid evolution. Regarding ontogeny, the innate component of language (...) capacity is likely to be polygenic and shared with other developmental domains. Dissociations between verbal and nonverbal development are probably rooted in the perceptuomotor specializations of neural substrates rather than the autonomy of a grammar module. Aphasiologicaldata often assumed to suggest modular linguistic subsystems can be accounted for in terms of a neurofunctional model incorporating perceptuomotor-based regional specializationsand distributivity of representations. Thus, dissociations between grammatical functors and content words are due to different conditions of acquisition and resulting differences in neural representation. Human brains are characterized by multifactorial interindividual variability, and strict universality of functional organization is biologically unrealistic.A theoretical alternative is proposed according to which (1) linguistic specialization of brain areas is due to epigenetic and probabilistic maturational events, not to genetic ”hard-wiring,” and (2) linguistic knowledge is neurally represented in distributed cell assemblies whose topography reflects the perceptuomotor modalities involved in the acquisition and use of a given item of knowledge. (shrink)

Learning theory has frequently been applied to language acquisition, but discussion has largely focused on information theoretic problems—in particular on the absence of direct negative evidence. Such arguments typically neglect the probabilistic nature of cognition and learning in general. We argue first that these arguments, and analyses based on them, suffer from a major flaw: they systematically conflate the hypothesis class and the learnable concept class. As a result, they do not allow one to draw significant conclusions about (...) the learner. Second, we claim that the real problem for language learning is the computational complexity of constructing a hypothesis from input data. Studying this problem allows for a more direct approach to the object of study—the language acquisition device—rather than the learnable class of languages, which is epiphenomenal and possibly hard to characterize. The learnability results informed by complexity studies are much more insightful. They strongly suggest that target grammars need to be objective, in the sense that the primitive elements of these grammars are based on objectively definable properties of the language itself. These considerations support the view that language acquisition proceeds primarily through data-driven learning of some form. (shrink)

Previous research (Palij & Aaronson, 1989) has shown that age of English acquisition (AEA) does not affect recall of English words even though SAT verbal scores systematically decrease with increasing AEA. We report an experiment using a forced-choice recognition task with an intervening verbal task based on the SAT. Correct recognition was unrelated to AEA while performance on the verbal task decreased as a function of AEA. We examine some mechanisms for this dissociation.

Negative conditional stimulus valence acquired during fear conditioning may enhance fear relapse and is difficult to remove as it extinguishes slowly and does not respond to the instruction that unconditional stimulus presentations will cease. We examined whether instructions targeting CS valence would be more effective. In Experiment 1, an image of one person was paired with an aversive US, while another was presented alone. After acquisition, participants were given positive information about the CS+ poser and negative information about the (...) CS− poser. Instructions reversed the pattern of differential CS valence present during acquisition and eliminated differential electrodermal responding. In Experiment 2, we compared positive and negative CS revaluation by providing positive/negative information about the CS+ and neutral information about CS−. After positive revaluation, differential valence was removed and differential electrodermal responding remained intact. After negative revaluation, differential valence was strengthened and differential electrodermal responding was eliminated. Unexpectedly, the instructions did not affect the reinstatement of differential electrodermal responding. (shrink)

This volume presents the state of the art of recent research on the acquisition of semantics. Covering topics ranging from infants' initial acquisition of word meaning to the more sophisticated mapping between structure and meaning in the syntax-semantics interface, and the relation between logical content and inferences on language meaning (semantics and pragmatics), the papers in this volume introduce the reader to the variety of ways in which children come to realize that semantic content is encoded in word (...) meaning (for example, in the event semantics of the verbal domain or the scope of logical operators), and at the level of the sentence, which requires the composition of semantic meaning. The authors represent some of the most established and promising researchers in this domain, demonstrating collective expertise in a range of methodologies and topics relevant to the acquisition of semantics. This volume will serve as a valuable resource for students and faculty, and junior and seasoned researchers alike. (shrink)

Machine learning (ML) techniques have become pervasive across a range of different applications, and are now widely used in areas as disparate as recidivism prediction, consumer credit-risk analysis, and insurance pricing. Likewise, in the physical world, ML models are critical components in autonomous agents such as robotic surgeons and self-driving cars. Among the many ethical dimensions that arise in the use of ML technology in such applications, analyzing morally permissible actions is both immediate and profound. For example, there is the (...) potential for learned algorithms to become biased against certain groups. More generally, in so much that the decisions of ML models impact society, both virtually (e.g., denying a loan) and physically (e.g., driving into a pedestrian), notions of accountability, blame and responsibility need to be carefully considered. In this article, we advocate for a two-pronged approach ethical decision-making enabled using rich models of autonomous agency: on the one hand, we need to draw on philosophical notions of such as beliefs, causes, effects and intentions, and look to formalise them, as attempted by the knowledge representation community, but on the other, from a computational perspective, such theories need to also address the problems of tractable reasoning and (probabilistic) knowledge acquisition. As a concrete instance of this tradeoff, we report on a few preliminary results that apply (propositional) tractable probabilistic models to problems in fair ML and automated reasoning of moral principles. Such models are compilation targets for certain types of knowledge representation languages, and can effectively reason in service some computational tasks. They can also be learned from data. Concretely, current evidence suggests that they are attractive structures for jointly addressing three fundamental challenges: reasoning about possible worlds + tractable computation + knowledge acquisition. Thus, these seems like a good starting point for modelling reasoning robots as part of the larger ecosystem where accountability and responsibility is understood more broadly. (shrink)

Several studies have testified to the importance of a responsive linguistic input for children’s language acquisition and development. In particular, maternal use of expansions, imitations, interpretations, and labels has been shown to promote both children’s language comprehension and production. From this perspective, the present study examined the semantically contingent linguistic input addressed to very preterm children’s comparing it to that directed to full-term children observed during a semi-structured play session when the children were 24 months of age. The relationships (...) between maternal contingent utterances and children’s communicative repertoires were also investigated. The main results showed that mothers of full-term children produced a higher proportion of semantically contingent utterances than those of very preterm children; moreover, this variable was associated with children’s more advanced communicative-linguistic outcomes. Overall, this study supports the interdependence between mothers’ use of certain linguistic strategies and children’s communicative-linguistic repertoire, extending this evidence to children born very preterm and suggesting the importance of considering the semantic contingency aspect of child-directed speech to support the communicative and linguistic development of these children. (shrink)