Figure 1: A pr ototyp ical exa mple of a three-layer feed forward network, used by Plunkett and M archm an (1 991 ) to simulate learning the past-tense of En glish verbs. The inpu t units encode representations of the three phonemes of the present tense of the artificial words used in this simulation. Th e netwo rk is trained to produce a representation of the phonemes employed in the past tense form and the suffix (/d/, /ed/, or /t/) (...) used on regular verbs. To run the network, each input unit is assigned an activation value o f 0 or 1 , dep ending on whethe r the featu re is present or not. Eac h input unit is conne cted to each of the 30 hidden units by a we ighted conn ection and p rovid es an inp ut to each hidden unit equal to the product of the input unit's activation and the weight. Each hidd en unit's activation is then determined by summing ov er the va lues co ming fro m each inp ut unit to deter mine a netinput, and then applying a non-linear function (e.g., the logistic function 1/(1+e^netinput). Th is whole proced ure is. (shrink)

Neuropsychological deficits have been widely used to elucidate normal cognitive functioning. Can patients with such deficits also be used to understand conscious visual experience? In this paper, we ask what it would be like to be a patient with apperceptive agnosia . Philosophical analyses of such questions have suggested that subjectively experiencing what another person experiences would be impossible. Although such roadblocks into the conscious experience of others exist, the experimental study of both patients and neurologically normal subjects can be (...) used to understand visual processing mechanisms. In order to understand the visual processes damaged in apperceptive agnosia, we first review this syndrome and present a case study of one such patient, patient J.W. We then review several theoretical accounts of apperceptive agnosia, and we conclude that studies of the patients themselves may not allow us to discriminate between the various explanations of the syndrome. To test these accounts, we have simulated apperceptive agnosia in neurologically normal subjects. The implications of our results for understanding both apperceptive agnosia and normal visual processing are discussed. (shrink)

We argue that existing learning algorithms are often poorly equipped to solve problems involving a certain type of important and widespread regularity that we call “type-2 regularity.” The solution in these cases is to trade achieved representation against computational search. We investigate several ways in which such a trade-off may be pursued including simple incremental learning, modular connectionism, and the developmental hypothesis of “representational redescription.”.

Since the publication of Fodor's (1983) The Modularity of Mind, there have been quite a few discussions of cognitive modularity among cognitive scientists. Generally, in those discussions, modularity means a property of specialized cognitive processes or a domain-specific body of information. In actuality, scholars understand modularity in many different ways. Different characterizations of modularity and modules were proposed and discussed, but they created misunderstanding and confusion. In this article, I classified and analyzed different approaches to modularity and argued for the (...) unity of modularity. Modularity is a multidimensional property consisting of features from several dimensions specifying different aspects of cognition. Among those, there are core features of modularity, and these core features form a cross-dimensional unity. Despite the diverse and liberal characterizations, modularity contributes to cognitive science because of the unity of the core features. (shrink)

In this paper I assess the explanatory role of internal representations in connectionist models of cognition. Focusing on both the internal‘hidden’units and the connection weights between units, I argue that the standard reasons for viewing these components as representations are inadequate to bestow an explanatorily useful notion of representation. Hence, nothing would be lost from connectionist accounts of cognitive processes if we were to stop viewing the weights and hidden units as internal representations.

In this paper I assess the explanatory role of internal representations in connectionist models of cognition. Focusing on both the internal‘hidden’units and the connection weights between units, I argue that the standard reasons for viewing these components as representations are inadequate to bestow an explanatorily useful notion of representation. Hence, nothing would be lost from connectionist accounts of cognitive processes if we were to stop viewing the weights and hidden units as internal representations.

Fodor has argued that observation is theory neutral, since the perceptual systems are modular, that is, they are domain‐specific, encapsulated, mandatory, fast, hard‐wired in the organism, and have a fixed neural architecture. Churchland attacks the theoretical neutrality of observation on the grounds that (a) the abundant top‐down pathways in the brain suggest the cognitive penetration of perception and (b) perceptual learning can change in the wiring of the perceptual systems. In this paper I introduce a distinction between sensation, perception, and (...) observation and I argue that although Churchland is right that observation involves top‐down processes, there is also a substantial amount of information in perception which is theory‐neutral. I argue that perceptual learning does not threaten the cognitive impenetrability of perception, and that the neuropsychological research does not provide evidence in favor of the top‐down character of perception. Finally, I discuss the possibility of an off‐line cognitive penetrability of perception. (shrink)

This article addresses the ability of Parallel Distributed Processing (PDP) networks to generate stagewise cognitive development in accordance with Piaget's theory of cognitive epigenesis. We carried out a replication study of the simulation experiments by McClelland (1989) and McClelland and Jenkins (1991) in which a PDP network learns to solve balance scale problems. In objective tests motivated from catastrophe theory, a mathematical theory of transitions in epigenetical systems, no evidence for stage transitions in network performance was found. It is concluded (...) that PDP networks lack the ability to recover the positive outcomes of analogous catastrophe analyses of real cognitive developmental data. In an attempt to further characterize the learning behaviour of PDP networks, we carried out a second simulation study using the discrimination‐shift paradigm. The results thus obtained indicate that PDP learning is compatible with the learning of stimulus‐response relationships, not with the acquisition of mediating rules such as conceived in (neo‐)Piagetian theory. In closing, we speculate about the feasibility of simulating stagewise development with alternative network architectures. (shrink)

A distributed neural network model called SPEC for processing sentences with recursive relative clauses is described. The model is based on separating the tasks of segmenting the input word sequence into clauses, forming the case‐role representations, and keeping track of the recursive embeddings into different modules. The system needs to be trained only with the basic sentence constructs, and it generalizes not only to new instances of familiar relative clause structures but to novel structures as well. SPEC exhibits plausible memory (...) degradation as the depth of the center embeddings increases, its memory is primed by earlier constituents, and its performance is aided by semantic constraints between the constituents. The ability to process structure is largely due to a central executive network that monitors and controls the execution of the entire system. This way, in contrast to earlier subsymbolic systems, parsing is modeled as a controlled high‐level process rather than one based on automatic reflex responses. (shrink)

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of (...) memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. (shrink)

The contributions to this special issue on cognitive development collectively propose ways in which learning involves developing constraints that shape subsequent learning. A learning system must be constrained to learn efficiently, but some of these constraints are themselves learnable. To know how something will behave, a learner must know what kind of thing it is. Although this has led previous researchers to argue for domain-specific constraints that are tied to different kinds/domains, an exciting possibility is that kinds/domains themselves can be (...) learned. General cognitive constraints, when combined with rich inputs, can establish domains, rather than these domains necessarily preexisting prior to learning. Knowledge is structured and richly differentiated, but its “skeleton” must not always be preestablished. Instead, the skeleton may be adapted to fit patterns of co-occurrence, task requirements, and goals. Finally, we argue that for models of development to demonstrate genuine cognitive novelty, it will be helpful for them to move beyond highly preprocessed and symbolic encodings that limit flexibility. We consider two physical models that learn to make tone discriminations. They are mechanistic models that preserve rich spatial, perceptual, dynamic, and concrete information, allowing them to form surprising new classes of hypotheses and encodings. (shrink)

Some regularities enjoy only an attenuated existence in a body of training data. These are regularities whose statistical visibility depends on some systematic recoding of the data. The space of possible recodings is, however, infinitely large – it is the space of applicable Turing machines. As a result, mappings that pivot on such attenuated regularities cannot, in general, be found by brute-force search. The class of problems that present such mappings we call the class of “type-2 problems.” Type-1 problems, by (...) contrast, present tractable problems of search insofar as the relevant regularities can be found by sampling the input data as originally coded. Type-2 problems, we suggest, present neither rare nor pathological cases. They are rife in biologically realistic settings and in domains ranging from simple animat behaviors to language acquisition. Not only are such problems rife – they are standardly solved! This presents a puzzle. How, given the statistical intractability of these type-2 cases, does nature turn the trick? One answer, which we do not pursue, is to suppose that evolution gifts us with exactly the right set of recoding biases so as to reduce specific type-2 problems to type-1 mappings. Such a heavy-duty nativism is no doubt sometimes plausible. But we believe there are other, more general mechanisms also at work. Such mechanisms provide general strategies for managing problems of type-2 complexity. Several such mechanisms are investigated. At the heart of each is a fundamental ploy – namely, the maximal exploitation of states of representation already achieved by prior, simpler learning so as to reduce the amount of subsequent computational search. Such exploitation both characterizes and helps make unitary sense of a diverse range of mechanisms. These include simple incremental learning, modular connectionism, and the developmental hypothesis of “representational redescription”. In addition, the most distinctive features of human cognition – language and culture – may themselves be viewed as adaptations enabling this representation/computation trade-off to be pursued on an even grander scale. (shrink)

The ability to combine words into novel sentences has been used to argue that humans have symbolic language production abilities. Critiques of connectionist models of language often center on the inability of these models to generalize symbolically (Fodor & Pylyshyn, 1988; Marcus, 1998). To address these issues, a connectionist model of sentence production was developed. The model had variables (role‐concept bindings) that were inspired by spatial representations (Landau & Jackendoff, 1993). In order to take advantage of these variables, a novel (...) dual‐pathway architecture with event semantics is proposed and shown to be better at symbolic generalization than several variants. This architecture has one pathway for mapping message content to words and a separate pathway that enforces sequencing constraints. Analysis of the model's hidden units demonstrated that the model learned different types of information in each pathway, and that the model's compositional behavior arose from the combination of these two pathways. The model's ability to balance symbolic and statistical behavior in syntax acquisition and to model aphasic double dissociations provided independent support for the dual‐pathway architecture. (shrink)

Modularity in the human brain remains a controversial issue, with disagreement over the nature of the modules that exist, and why, when, and how they emerge. It is a natural assumption that modularity offers some form of computational advantage, and hence evolution by natural selection has translated those advantages into the kind of modular neural structures familiar to cognitive scientists. However, simulations of the evolution of simplified neural systems have shown that, in many cases, it is actually non-modular architectures that (...) are most efficient. In this paper, the relevant issues are discussed and a series of simulations are presented that reveal crucial dependencies on the details of the learning algorithms and tasks that are being modelled, and the importance of taking into account known physical brain constraints, such as the degree of neural connectivity. A pattern is established which provides one explanation of why modularity should emerge reliably across a range of neural processing tasks. (shrink)

The notion of levels has been widely used in discussions of cognitive science, especially in discussions of the relation of connectionism to symbolic modeling of cognition. I argue that many of the notions of levels employed are problematic for this purpose, and develop an alternative notion grounded in the framework of mechanistic explanation. By considering the source of the analogies underlying both symbolic modeling and connectionist modeling, I argue that neither is likely to provide an adequate analysis of processes at (...) the level at which cognitive theories attempt to function: One is drawn from too low a level, the other from too high a level. If there is a distinctly cognitive level, then we still need to determine what are the basic organizational principles at that level. (shrink)

The claim of the multiple realizability of mental states by brain states has been a major feature of the dominant philosophy of mind of the late 20th century. The claim is usually motivated by evidence that mental states are multiply realized, both within humans and between humans and other species. We challenge this contention by focusing on how neuroscientists differentiate brain areas. The fact that they rely centrally on psychological measures in mapping the brain and do so in a comparative (...) fashion undercuts the likelihood that, at least within organic life forms, we are likely to find cases of multiply realized psychological functions. (shrink)

This paper reviews four significant advances on the feedforward architecture that has dominated discussions of connectionism. The first involves introducing modularity into networks by employing procedures whereby different networks learn to perform different components of a task, and a Gating Network determines which network is best equiped to respond to a given input. The second consists in the use of recurrent inputs whereby information from a previous cycle of processing is made available on later cycles. The third development involves developing (...) compressed representations of strings in which there is no longer an explicit encoding of the components but where information about the structure of the original string can be recovered and so is present functionally. The final advance entails using connectionist learning procedures not just to change weights in networks but to change the patterns used as inputs to the network. These advances significantly increase the usefulness of connectionist networks for modeling human cognitive performance by, among other things, providing tools for explaining the productivity and systematicity of some mental activities, and developing representations that are sensitive to the content they are to represent. (shrink)