A central methodological innovation in cognitive science has been the development of connectionist or neural network models of psychological phenomena. These models may also comprise a theoretically integrative and methodologically rigorous approach to psychiatric phenomena. In this paper we employ connectionist theory to conceptualize obsessive-compulsive disorder . We discuss salient phenomenological and neurobiological findings of the illness, and then reformulate these using neural network models. Several features and mechanisms of OCD may be explicated in (...) terms of disordered networks. Neural network modeling appears to constitute a novel and potentially fertile approach to psychiatric disorders such as OCD. (shrink)

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine, a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning (...) and can encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models. We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. (shrink)

Clahsen's claim to contribute arguments for dual mechanisms based on rule analysis and against connectionist proposals is refuted. Both types of modeling are inadequate for principled reasons.

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly elegant and (...) powerful models of memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. (shrink)

Connectionism—also known as parallel distributed processing, or neural network modeling—offers promise as a framework to unite clinical and cognitive psychology, and as a tool for studying conscious and unconscious mental activity. This paper describes a neural network model of the case study of Lucy R., from Freud and Breuer's Studies on Hysteria. Though very simple in architecture, the network spontaneously displays analogues of repression and hallucination, corresponding to Lucy R.'s symptoms. Salient elements of Lucy's (...) conscious experience are represented in the model by the activations of neuronlike processors in a fully interconnected network, without hidden units. The model learns to associate elements of experience that were associated in Lucy's case history. Some of these configurations of elements were traumatic for Lucy; trauma is modeled in the network by "emphatic learning," learning accomplished at an abnormally high learning rate. The model suggests that changing associations among conscious elements are sufficient to generate the symptoms Freud observed: apparent repression, hallucination, and recovery through therapy. In the case of Lucy R., Freud's theoretical inference regarding active but unconscious thought is not required by his data. Instead, the unconscious can be understood as a set of complex dispositions embodied in connections between elements of conscious experience. (shrink)

This is an overview of recent philosophical discussion about connectionism and the foundations of cognitive science. Connectionist modeling in cognitive science is described. Three broad conceptions of the mind are characterized, and their comparative strengths and weaknesses are discussed: (1) the classical computation conception in cognitive science; (2) a popular foundational interpretation of connectionism that John Tienson and I call “non‐sentential computationalism”; and (3) an alternative interpretation of connectionism we call “dynamical cognition.” Also discussed are two recent philosophical (...) attempts to enlist connectionism in defense of eliminativism about folk psychology. (shrink)

Human participants and recurrent (“connectionist”) neural networks were both trained on a categorization system abstractly similar to natural language systems involving irregular (“strong”) classes and a default class. Both the humans and the networks exhibited staged learning and a generalization pattern reminiscent of the Elsewhere Condition (Kiparsky, 1973). Previous connectionist accounts of related phenomena have often been vague about the nature of the networks’ encoding systems. We analyzed our network using dynamical systems theory, revealing topological and (...) geometric properties that can be directly compared with the mechanisms of non-connectionist, rule-based accounts. The results reveal that the networks “contain” structures related to mechanisms posited by rule-based models, partly vindicating the insights of these models. On the other hand, they support the one mechanism (OM), as opposed to the more than one mechanism (MOM), view of symbolic abstraction by showing how the appearance of MOM behavior can arise emergently from one underlying set of principles. The key new contribution of this study is to show that dynamical systems theory can allow us to explicitly characterize the relationship between the two perspectives in implemented models. (shrink)

This paper reviews four significant advances on the feedforward architecture that has dominated discussions of connectionism. The first involves introducing modularity into networks by employing procedures whereby different networks learn to perform different components of a task, and a Gating Network determines which network is best equiped to respond to a given input. The second consists in the use of recurrent inputs whereby information from a previous cycle of processing is made available on later cycles. The third development (...) involves developing compressed representations of strings in which there is no longer an explicit encoding of the components but where information about the structure of the original string can be recovered and so is present functionally. The final advance entails using connectionist learning procedures not just to change weights in networks but to change the patterns used as inputs to the network. These advances significantly increase the usefulness of connectionist networks for modeling human cognitive performance by, among other things, providing tools for explaining the productivity and systematicity of some mental activities, and developing representations that are sensitive to the content they are to represent. (shrink)

The contents and structure of semantic memory have been the focus of much recent research, with major advances in the development of distributional models, which use word co‐occurrence information as a window into the semantics of language. In parallel, connectionist modeling has extended our knowledge of the processes engaged in semantic activation. However, these two lines of investigation have rarely been brought together. Here, we describe a processing model based on distributional semantics in which activation spreads throughout a (...) semantic network, as dictated by the patterns of semantic similarity between words. We show that the activation profile of the network, measured at various time points, can successfully account for response times in lexical and semantic decision tasks, as well as for subjective concreteness and imageability ratings. We also show that the dynamics of the network is predictive of performance in relational semantic tasks, such as similarity/relatedness rating. Our results indicate that bringing together distributional semantic networks and spreading of activation provides a good fit to both automatic lexical processing (as indexed by lexical and semantic decisions) as well as more deliberate processing (as indexed by ratings), above and beyond what has been reported for previous models that take into account only similarity resulting from network structure. (shrink)

A set of hypotheses is formulated for a connectionist approach to cognitive modeling. These hypotheses are shown to be incompatible with the hypotheses underlying traditional cognitive models. The connectionist models considered are massively parallel numerical computational systems that are a kind of continuous dynamical system. The numerical variables in the system correspond semantically to fine-grained features below the level of the concepts consciously used to describe the task domain. The level of analysis is intermediate between those of (...) symbolic cognitive models and neural models. The explanations of behavior provided are like those traditional in the physical sciences, unlike the explanations provided by symbolic models.Higher-level analyses of these connectionist models reveal subtle relations to symbolic models. Parallel connectionist memory and linguistic processes are hypothesized to give rise to processes that are describable at a higher level as sequential rule application. At the lower level, computation has the character of massively parallel satisfaction of soft numerical constraints; at the higher level, this can lead to competence characterizable by hard rules. Performance will typically deviate from this competence since behavior is achieved not by interpreting hard rules but by satisfying soft constraints. The result is a picture in which traditional and connectionist theoretical constructs collaborate intimately to provide an understanding of cognition. (shrink)

Connectionism is a style of modeling based upon networks of interconnected simple processing devices. This style of modeling goes by a number of other names too. Connectionist models are also sometimes referred to as 'Parallel Distributed Processing' (or PDP for short) models or networks.1 Connectionist systems are also sometimes referred to as 'neural networks' (abbreviated to NNs) or 'artificial neural networks' (abbreviated to ANNs). Although there may be some rhetorical appeal to this neural (...) nomenclature, it is in fact misleading as connectionist networks are commonly significantly dissimilar to neurological systems. For this reason, I will avoid using this terminology, other than in direct quotations. Instead, I will follow the practice I have adopted above and use 'connectionist' as my primary term for systems of this kind. (shrink)

Evidence from numerous studies using the visual world paradigm has revealed both that spoken language can rapidly guide attention in a related visual scene and that scene information can immediately influence comprehension processes. These findings motivated the coordinated interplay account (Knoeferle & Crocker, 2006) of situated comprehension, which claims that utterance‐mediated attention crucially underlies this closely coordinated interaction of language and scene processing. We present a recurrent sigma‐pi neural network that models the rapid use of scene information, exploiting (...) an utterance‐mediated attentional mechanism that directly instantiates the CIA. The model is shown to achieve high levels of performance (both with and without scene contexts), while also exhibiting hallmark behaviors of situated comprehension, such as incremental processing, anticipation of appropriate role fillers, as well as the immediate use, and priority, of depicted event information through the coordinated use of utterance‐mediated attention to the scene. (shrink)

Missing from Quartz & Sejnowski's (Q&S's) unique and valuable effort to relate cognitive development to neural constructivism is an examination of the global emergent properties of adding new neural circuits. Such emergent properties can be studied with computational models. Modeling with generative connectionist networks shows that synaptogenic mechanisms can account for progressive increases in children's representational power.

Several approaches to implementing symbol-like representations in neurally plausible models have been proposed. These approaches include binding through synchrony, “mesh” binding, and conjunctive binding. Recent theoretical work has suggested that most of these methods will not scale well, that is, that they cannot encode structured representations using any of the tens of thousands of terms in the adult lexicon without making implausible resource assumptions. Here, we empirically demonstrate that the biologically plausible structured representations employed in the Semantic Pointer Architecture approach (...) to modeling cognition do scale appropriately. Specifically, we construct a spiking neural network of about 2.5 million neurons that employs semantic pointers to successfully encode and decode the main lexical relations in WordNet, which has over 100,000 terms. In addition, we show that the same representations can be employed to construct recursively structured sentences consisting of arbitrary WordNet concepts, while preserving the original lexical structure. We argue that these results suggest that semantic pointers are uniquely well-suited to providing a biologically plausible account of the structured representations that underwrite human cognition. (shrink)

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely on a (...) dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP. (shrink)

The representation of knowledge in the law has basically followed a rule-based logical-symbolic paradigm. This paper aims to show how the modeling of legal knowledge can be re-examined using connectionist models, from the perspective of the theory of the dynamics of unstable systems and chaos. We begin by showing the nature of the paradigm shift from a rule-based approach to one based on dynamic structures and by discussing how this would translate into the field of theory of law. (...) In order to show the full potential of this new approach, we start from an experiment with NEUROLEX, in which a neural network was used to model a corpus of French Council of State decisions. We examine the implications of this experiment, especially those concerning the limits of the model used, and show that other connectionist models might correspond more adequately to the nature of legal knowledge. Finally, we propose another neural model which could show not only the rules which emerge from legal qualification (NEUROLEX's goal), but also the way in which a legal qualification process evolves from one concept to another. (shrink)

An influential view of the nature of the language system is that of an evolved biological system in which a set of rules is combined with a lexicon that contains the words of the language together with a representation of their context. Alternative views, usually based on connectionist modeling, attempt to explain the structure of language on the basis of complex associative processes. Here, I put forward a third view that stresses experience-dependent structural development of the brain circuits (...) supporting language as a core principle of the organization of the language system. In this view, embodied in a recent neuroconstructivist neural network of past tense development and processing, initial domain-general predispositions enable the development of functionally specialized brain structures through interactions between experience-dependent brain development and statistical learning in a structured environment. Together, these processes shape a biological adult language system that appears to separate into distinct mechanism for processing rules and exceptions, whereas in reality those subsystems co-develop and interact closely. This view puts experience-dependent brain development in response to a specific language environment at the heart of understanding not only language development but adult language processing as well. (shrink)

This paper is a commentary on the target article by Michael Arbib, “Levels of modeling of mechanisms of visually guided behavior”, in the same issue of the journal, pp. 407–465. -/- I focus on the importance of the inclusion of an ability of a system to entertain, at a given time, multiple instantiations of a given schema (situation template, frame, script, action plan, etc.), and complications introduced into neural/connectionist network systems by such inclusion.

Structure–activity relationship and quantitative SAR are modeling methods largely used in assessing biological properties of chemical substances. QSAR is based on the hypothesis that the chemical structure is responsible for the activity; it follows that similar molecules are expected to have similar properties. Similarity plays an important role in read across, which categorizes molecules primarily on the basis of similarity. Similarity, and chemical similarity too, is a property differently perceived by humans. The various proposed metrics often disagree with human (...) judgment, and no a unique metric for chemical similarity is universally adopted. Researchers argued that categorization is not only explained by similarity but depends as well on abstract knowledge and the task to accomplish. Moreover, similarity cannot be the unique explanation of a categorization, as different perceptual processes take place in category formation. Assuming that similarity judgments are deeply rooted in human knowledge and perception, cognitive sciences contributions are as important as the mathematical considerations of the classical theories. After an excursus in the many views of similarity in philosophy, mathematics, and cognitive science, the paper explores how connectionist systems, which loosely mimic the human cognitive system, could improve similarity-based choices. A case study on building SARs using connectionism and deep neural networks shows the role of similarity in building and explaining those models. A discussion about deep learning for QSAR and as a modeling tool for science concludes the presentation. (shrink)

Formal models of argumentation have been investigated in several areas, from multi-agent systems and artificial intelligence (AI) to decision making, philosophy and law. In artificial intelligence, logic-based models have been the standard for the representation of argumentative reasoning. More recently, the standard logic-based models have been shown equivalent to standard connectionist models. This has created a new line of research where (i) neural networks can be used as a parallel computational model for argumentation and (ii) neural networks (...) can be used to combine argumentation, quantitative reasoning and statistical learning. At the same time, non-standard logic models of argumentation started to emerge. In this paper, we propose a connectionist cognitive model of argumentation that accounts for both standard and non-standard forms of argumentation. The model is shown to be an adequate framework for dealing with standard and non-standard argumentation, including joint-attacks, argument support, ordered attacks, disjunctive attacks, meta-level attacks, self-defeating attacks, argument accrual and uncertainty. We show that the neural cognitive approach offers an adequate way of modelling all of these different aspects of argumentation. We have applied the framework to the modelling of a public prosecution charging decision as part of a real legal decision making case study containing many of the above aspects of argumentation. The results show that the model can be a useful tool in the analysis of legal decision making, including the analysis of what-if questions and the analysis of alternative conclusions. The approach opens up two new perspectives in the short-term: the use of neural networks for computing prevailing arguments efficiently through the propagation in parallel of neuronal activations, and the use of the same networks to evolve the structure of the argumentation network through learning (e.g. to learn the strength of arguments from data). (shrink)

Language acquisition may be one of the most difficult tasks that children face during development. They have to segment words from fluent speech, figure out the meanings of these words, and discover the syntactic constraints for joining them together into meaningful sentences. Over the past couple of decades, computational modeling has emerged as a new paradigm for gaining insights into the mechanisms by which children may accomplish these feats. Unfortunately, many of these models assume a computational complexity and linguistic (...) knowledge likely to be beyond the abilities of developing young children. This article shows that, using simple statistical procedures, significant correlations exist between the beginnings and endings of a word and its lexical category in English, Dutch, French, and Japanese. Therefore, phonetic information can contribute to individuating higher level structural properties of these languages. This article also presents a simple 2‐layer connectionist model that, once trained with an initial small sample of words labeled for lexical category, can infer the lexical category of a large proportion of novel words using only word‐edge phonological information, namely the first and last phoneme of a word. The results suggest that simple procedures combined with phonetic information perceptually available to children provide solid scaffolding for emerging lexical categories in language development. (shrink)

There is a vast literature within philosophy of mind that focuses on artificial intelligence, but hardly mentions methodological questions. There is also a growing body of work in philosophy of science about modeling methodology that hardly mentions examples from cognitive science. Here these discussions are connected. Insights developed in the philosophy of science literature about the importance of idealization provide a way of understanding the neural implausibility of connectionist networks. Insights from neurocognitive science illuminate how relevant similarities (...) between models and targets are picked out, how modeling inferences are justified, and the metaphysical status of models. (shrink)

What are the processes, from conception to adulthood, that enable a single cell to grow into a sentient adult? The processes that occur along the way are so complex that any attempt to understand development necessitates a multi-disciplinary approach, integrating data from cognitive studies, computational work, and neuroimaging - an approach till now seldom taken in the study of child development. Neuroconstructivism is a major new 2 volume publication that seeks to redress this balance, presenting an integrative new framework for (...) considering development. In the first volume, the authors review up-to-to date findings from neurobiology, brain imaging, child development, computer and robotic modelling to consider why children's thinking develops the way it does. They propose a new synthesis of development that is based on 5 key principles found to operate at many levels of descriptions. They use these principles to explain what causes a number of key developmental phenomena, including infants' interacting with objects, early social cognitive interactions, and the causes of dyslexia. The "neuroconstructivist" framework also shows how developmental disorders do not arise from selective damage to normal cognitive systems, but instead arise from developmental processes that operate under atypical constraints. How these principles work is illustrated in several case studies ranging from perceptual to social and reading development. Finally, the authors use neuroimaging, behavioural analyses, computational simulations and robotic models to provide a way of understanding the mechanisms and processes that cause development to occur. Computer and robotic models provide concrete tools for investigating the processes and mechanisms involved in learning and development. Volume 2 illustrates the principles of 'Neuroconstructivist' development, with contributions from 9 different labs across the world. Each of the contributions illustrates how models play a central role in understanding development. The models presented include standard connectionist neural network models as well as multi-agent models. Also included are robotic models emphasizing the need to take embodiment and brain-system interactions seriously. A model of Autism and one of Specific Language Impairment also illustrate how atypical development can be understood in terms of the typical processes of development but operating under restricted conditions. This volume complements Volume 1 by providing concrete examples of how the 'Neuroconstructivist' principles can be grounded within a diverse range of domains, thereby shaping the research agenda in those domains. (shrink)

One's model of skill determines what one expects from neural network modelling and how one proposes to go about enhancing expertise. We view skill acquisition as a progression from acting on the basis of a rough theory of a domain in terms of facts and rules to being able to respond appropriately to the current situation on the basis of neuron connections changed by the results of responses to the relevant aspects of many past situations. Viewing skill acquisition (...) in this ways suggests how one can avoid the problem currently facing AI of how to train a network to make human-like generalizations. In training a network one must progress, as the human learner does, from rules and facts to wholistic responses. As to future work, from our perspective one should not try to enhance expertise as in traditional AI by attempting to construct improved theories of a domain, but rather by improving the learner's access to the relevant aspects of a domain so as to facilitate learning from experience. (shrink)