There is disagreement over the notion of representation in cognitive science. Many investigators equate representations with symbols, that is, with syntactically defined elements in an internal symbol system. In recent years there have been two challenges to this orthodoxy. First, a number of philosophers, including many outside the symbolist orthodoxy, have argued that "representation" should be understood in its classical sense, as denoting a "stands for" relation between representation and represented. Second, there has been a growing challenge to orthodoxy (...) under the banner of connectionism. Although this connectionist challenge has evoked emotionally-charged rebukes from the symbolist camp, connectionists as a group have not articulated a conception of representation to replace the symbolist view. Nonetheless, most agree on the need for a nonsymbolic notion of representation. This paper advances a "stands for" sense of representation as primary by incorporating it into a general approach to cognitive science consonant with the connectionism. The idea is to marry connectionism to a particular version of functionalism, viz., one that builds its notion of "function" on the similarity between functional analysis in biology and in psychology. It builds on my earlier work adapting and revising Marr's tri-level approach to cognition. My proposal frees Marr's analysis from its moorings in the orthodox symbolic view of representation and allies it with a notion of functional analysis akin to that proposed by Cummins, developed further by Haugeland, and invoked by Millikan and Dretske. Unlike these authors, however, I do not wed representation to a general belief-desire analysis of behavior. Rather, I follow what I take to be the lesson of psychological practice, according to which the investigator does not seek to explain behavior in general but seeks to analyze the cognitive capacities that underlie behavior, such as vision, memory, learning, and linguistic capacities. Accordingly, ascriptions of representational content are made not by working back from belief-desire ascriptions, but in the context of forming psychological models to account for specific cognitive capacities. Because conceptions of representation in cognitive science typically are embedded in a general approach to the study of cognition, arguments for the comparative plausibility of a particular conception of representation must scout these larger frameworks. I therefore begin by characterizing the interlocking set of assumptions that gave life to the orthodox symbolist approach, paying special attention to the complementarity between representation and process that naturally arises from those assumptions. I then consider two versions of an alternative approach to representation: Dretske's and my own. Finally, I urge the merits of the "cognitive capacities" over the "belief-desire" approach to the subject-matter of cognitive science. (shrink)

The relation between logic and thought has long been controversial, but has recently influenced theorizing about the nature of mental processes in cognitive science. One prominent tradition argues that to explain the systematicity of thought we must posit syntactically structured representations inside the cognitive system which can be operated upon by structure sensitive rules similar to those employed in systems of natural deduction. I have argued elsewhere that the systematicity of human thought might better be explained as resulting from (...) the fact that we have learned natural languages which are themselves syntactically structured. According to this view, symbols of natural language are external to the cognitive processing system and what the cognitive system must learn to do is produce and comprehend such symbols. In this paper I pursue that idea by arguing that ability in natural deduction itself may rely on pattern recognition abilities that enable us to operate on external symbols rather than encodings of rules that might be applied to internal representations. To support this suggestion, I present a series of experiments with connectionist networks that have been trained to construct simple natural deductions in sentential logic. These networks not only succeed in reconstructing the derivations on which they have been trained, but in constructing new derivations that are only similar to the ones on which they have been trained. (shrink)

Not long ago the standard view in cognitive science was that representations are symbols in an internal representational system or language of thought and that psychological processes are computations defined over such representations. This orthodoxy has been challenged by adherents of functional analysis and by connectionists. Functional analysis as practiced by Marr is consistent with an analysis of representation that grants primacy to a stands for conception of representation. Connectionism is also compatible with this notion of representation; when conjoined with (...) functional analysis, it provides a means of analyzing psychological systems in term of rules and representations without becoming committed to symbolism. Direct theorists, who rejected the orthodox symbolist conception of representation because it violated their strictures against cognitive mediational mechanisms, should find it possible to accept rules-and-representations and information-processing analyses of the mechanisms of information pickup couched in terms of functional analysis. (shrink)

The paper is a commentary on the target article by Christine A. Skarda & Walter J. Freeman, “How brains make chaos in order to make sense of the world”, in the same issue of the journal, pp.161–195. -/- I confine my comments largely to some philosophical claims that Skarda & Freeman make and to the relationship of their model to connectionism. Some of the comments hinge on what symbols are and how they might sit in neural systems.

[Edited from Conclusion section:] We have looked at various challenging issues to do with getting connectionism to cope with high-level cognitive activities such a reasoning and natural language understanding. The issues are to do with various facets of generalization that are not commonly noted. We have been concerned in particular with the special forms these issues take in the arena of propositional attitude processing. The main problems we have looked at are: (1) The need to construct explicit representations of (...) generalizations, not just generalize correctly to individual cases; (2) The need to be able to match two or more complex short-term information structures, to enable rapid generalization from recent examples rather than from long-term memories; (3) The need to represent and reason with anomalous combinations of concepts; (4) The need to perform embedded reasoning. This presents special problems for systems using non-concatenative representations (as in mainstream connectionist approaches). We also touched on vague quantification in attitude report complements. Neither this topic nor that of analogies between short-term structures (point 2) has been adequately addressed in the symbolic framework, let alone in connectionism. -/- The opportunities and problems covered are put forward as things worth being optimistic about or pessimistic about, respectively. They are not put forward as decisive arguments for or against connectionism. The hope is that this chapter contributes to a greater understanding of the connectionist/symbolist gap by presenting some unusual issues and by throwing new light on some well known ones. (shrink)

Cognitive science is, more than anything else, a pursuit of cognitive mechanisms. To make headway towards a mechanistic account of any particular cognitive phenomenon, a researcher must choose among the many architectures available to guide and constrain the account. It is thus fitting that this volume on contemporary debates in cognitive science includes two issues of architecture, each articulated in the 1980s but still unresolved: " • Just how modular is the mind? – a debate initially pitting encapsulated mechanisms against (...) highly interactive ones. • Does the mind process language-like representations according to formal rules? – a debate initially pitting symbolic architectures against less language-like architectures. " Our project here is to consider the second issue within the broader context of where cognitive science has been and where it is headed. The notion that cognition in general—not just language processing—involves rules operating on language-like representations actually predates cognitive science. In traditional philosophy of mind, mental life is construed as involving propositional attitudes—that is, such attitudes towards propositions as believing, fearing, and desiring that they be true—and logical inferences from them. On this view, if a person desires that a proposition be true and believes that if she performs a certain action it will become true, she will make the inference and perform the action. (shrink)

This paper is devoted to some generalizations of explanatory potential of connectionist approaches to theoretical problems of the philosophy of mind. Are considered both strong, and weaknesses of neural network models. Connectionism has close methodological ties with modern neurosciences and neurophilosophy. And this fact strengthens its positions, in terms of empirical naturalistic approaches. However, at the same time this direction inherits weaknesses of computational approach, and in this case all system of anticomputational critical arguments becomes applicable to the connectionst (...) models of mind. The last developments in the field of deep learning gave rich empirical material for cognitive sciences. Multilayered networks, mathematical models of associative dynamics of learning, self-organizing neuronets and all that allow to explain the principles of human conceptual organizing and after this to emulate these processes in computer systems. At all engineering achievements of this technology there is a traditional criticism from representatives of cognitive psychology who cannot accept a thesis about learning ability of a neuronet on the basis of redistribution of scales. Process of learning of natural intelligence, according to cognitive models, happens due to attraction of knowledge broadcast in a symbolical form (mental representations, concepts) at the expense of the systems of output knowledge expressed in the propositional contents. Some philosophical aspects of «neural metaphor» in modern cognitive sciences create the problem field which demands comprehensive understanding, the first step towards which is taken in this work. (shrink)

A set of hypotheses is formulated for a connectionist approach to cognitive modeling. These hypotheses are shown to be incompatible with the hypotheses underlying traditional cognitive models. The connectionist models considered are massively parallel numerical computational systems that are a kind of continuous dynamical system. The numerical variables in the system correspond semantically to fine-grained features below the level of the concepts consciously used to describe the task domain. The level of analysis is intermediate between those of (...) symbolic cognitive models and neural models. The explanations of behavior provided are like those traditional in the physical sciences, unlike the explanations provided by symbolic models.Higher-level analyses of these connectionist models reveal subtle relations to symbolic models. Parallel connectionist memory and linguistic processes are hypothesized to give rise to processes that are describable at a higher level as sequential rule application. At the lower level, computation has the character of massively parallel satisfaction of soft numerical constraints; at the higher level, this can lead to competence characterizable by hard rules. Performance will typically deviate from this competence since behavior is achieved not by interpreting hard rules but by satisfying soft constraints. The result is a picture in which traditional and connectionist theoretical constructs collaborate intimately to provide an understanding of cognition. (shrink)

Something remarkable is happening in the cognitive sciences. After a quarter of a century of cognitive models that were inspired by the metaphor of the digital computer, the newest cognitive models are inspired by the properties of the brain itself. Variously referred to as connectionist, parallel distributed processing, or neutral network models, they explore the idea that complex intellectual operations can be carried out by large networks of simple, neuron-like units. The units themselves are identical, very low-level and (...) 'stupid'. Intelligent performance is derived from the pattern of connection strengths between units, and the fundamental cognitive activity is pattern recognition and completion. Connectionism and the Mind provides an introduction to this newly emerging approach to understanding the mind. The first few chapters focus on network architecture, offering accessible treatment of the equations that describe learning and the propagation of activation. Furthermore, the reader is walked step-by-step through the activities of networks engaged in pattern recognition, learning, and cognitive tasks such as memory retrieval and prototype formation. The remainder of the book addresses the implications of connectionism for theories of the mind, both philosophical and psychological. Foe example: What Role is played by pattern recognition and completion as basic as cognitive functions? Connectionist models have particular strength in learning and pattern recognition; should they be limited to those functions, or can they provide an overall account of cognitive functioning? In particular, can connectionist models provide an adequate account of the ability to employ linguistic and other symbol systems, or must an adequate system incorporate symbol processing as a basic cognitive capacity? Finally, Connectionism and the Mind examines the relation of connectionist models to philosophical accounts of propositional attitudes, and to a variety of other inquiries in cognitive psychology, linguistics, developmental psychology, artificial intelligence and neuroscience. (shrink)

The philosophy of cognitive science has recently become one of the most exciting and fastest growing domains of philosophical inquiry and analysis. Until the early 1980s, nearly all of the models developed treated cognitive processes -- like problem solving, language comprehension, memory, and higher visual processing -- as rule-governed symbol manipulation. However, this situation has changed dramatically over the last half dozen years. In that period there has been an enormous shift of attention toward connectionist models of (...) cognition that are inspired by the network-like architecture of the brain. Because of their unique architecture and style of processing, connectionist systems are generally regarded as radically different from the more traditional symbol manipulation models. This collection was designed to provide philosophers who have been working in the area of cognitive science with a forum for expressing their views on these recent developments. Because the symbol-manipulating paradigm has been so important to the work of contemporary philosophers, many have watched the emergence of connectionism with considerable interest. The contributors take very different stands toward connectionism, but all agree that the potential exists for a radical shift in the way many philosophers think of various aspects of cognition. Exploring this potential and other philosophical dimensions of connectionist research is the aim of this volume. (shrink)

There is disagreement over the notion of representation in cognitive science. Many investigators equate representations with symbols, that is, with syntactically defined elements in an internal symbol system. In recent years there have been two challenges to this orthodoxy. First, a number of philosophers, including many outside the symbolist orthodoxy, have argued that "representation" should be understood in its classical sense, as denoting a "stands for" relation between representation and represented. Second, there has been a growing challenge to orthodoxy (...) under the banner of connectionism. Although this connectionist challenge has evoked emotionally-charged rebukes from the symbolist camp, connectionists as a group have not articulated a conception of representation to replace the symbolist view. Nonetheless, most agree on the need for a nonsymbolic notion of representation. This paper advances a "stands for" sense of representation as primary by incorporating it into a general approach to cognitive science consonant with the connectionism. The idea is to marry connectionism to a particular version of functionalism, viz., one that builds its notion of "function" on the similarity between functional analysis in biology and in psychology. It builds on my earlier work adapting and revising Marr's tri-level approach to cognition. My proposal frees Marr's analysis from its moorings in the orthodox symbolic view of representation and allies it with a notion of functional analysis akin to that proposed by Cummins, developed further by Haugeland, and invoked by Millikan and Dretske. Unlike these authors, however, I do not wed representation to a general belief-desire analysis of behavior. Rather, I follow what I take to be the lesson of psychological practice, according to which the investigator does not seek to explain behavior in general but seeks to analyze the cognitive capacities that underlie behavior, such as vision, memory, learning, and linguistic capacities. Accordingly, ascriptions of representational content are made not by working back from belief-desire ascriptions, but in the context of forming psychological models to account for specific cognitive capacities. Because conceptions of representation in cognitive science typically are embedded in a general approach to the study of cognition, arguments for the comparative plausibility of a particular conception of representation must scout these larger frameworks. I therefore begin by characterizing the interlocking set of assumptions that gave life to the orthodox symbolist approach, paying special attention to the complementarity between representation and process that naturally arises from those assumptions. I then consider two versions of an alternative approach to representation: Dretske's and my own. Finally, I urge the merits of the "cognitive capacities" over the "belief-desire" approach to the subject-matter of cognitive science. (shrink)

The concept of representation has become almost inextricably bound to the concept of symbol systems. the concepts is nowhere more prevalent than in descriptions of "internal representations." These representations are thought to occur in an internal symbol system that allows the brain to store and use information. In this paper we explore a different approach to understanding psychological processes, one that retains a commitment to representations and computations but that is not based on the idea that information (...) must be stored and manipulated in symbol systems. In particular, we suggest that the notion of a symbol system as currently understood construes psychological processes in terms of a specific type of computational system, in which a control function "reads," "interprets," and manipulates discrete entities called "symbols." We argue that other types of computational systems may provide a more appropriate characterization of psychological processes. One implication of our argument is the need to consider the constraints placed on computational theories in psychology by the nature of the computing device itself, the human brain. Perhaps surprisingly, this implication leads us to the conclusion that a "functionalist" conception of psychological processes (discussed below) does not entail that physiology is irrelevant to psychology, as has been maintained by prominent adherents of the symbol-systems approach. (shrink)

In this paper we defend a position we call radical connectionism. Radical connectionism claims that cognition _never_ implicates an internal symbolic medium, not even when natural language plays a part in our thought processes. On the face of it, such a position renders the human capacity for abstract thought quite mysterious. However, we argue that connectionism is committed to an analog conception of neural computation, and that representation of the abstract is no more problematic for a system of analog vehicles (...) than for a symbol system. Natural language is therefore not required as a representational medium for abstract thought. Since natural language is arguably not a representational medium _at all_, but a conventionally governed scheme of communicative signals, we suggest that the role of internalised (i.e., self- directed) language is best conceived in terms of the coordination and control of cognitive activities within the brain. (shrink)

Abduction is or subsumes a process of inference. It entertains possible hypotheses and it chooses hypotheses for further scrutiny. There is a large literature on various aspects of non-symbolic, subconscious abduction. There is also a very active research community working on the symbolic (logical) characterisation of abduction, which typically treats it as a form of hypothetico-deductive reasoning. In this paper we start to bridge the gap between the symbolic and sub-symbolic approaches to abduction. We are interested in benefiting from developments (...) made by each community. In particular, we are interested in the ability of non-symbolic systems (neural networks) to learn from experience using efficient algorithms and to perform massively parallel computations of alternative abductive explanations. At the same time, we would like to benefit from the rigour and semantic clarity of symbolic logic. We present two approaches to dealing with abduction in neural networks. One of them uses Connectionist Modal Logic and a translation of Horn clauses into modal clauses to come up with a neural network ensemble that computes abductive explanations in a top-down fashion. The other combines neural-symbolic systems and abductive logic programming and proposes a neural architecture which performs a more systematic, bottom-up computation of alternative abductive explanations. Both approaches employ standard neural network architectures which are already known to be highly effective in practical learning applications. Differently from previous work in the area, our aim is to promote the integration of reasoning and learning in a way that the neural network provides the machinery for cognitive computation, inductive learning and hypothetical reasoning, while logic provides the rigour and explanation capability to the systems, facilitating the interaction with the outside world. Although it is left as future work to determine whether the structure of one of the proposed approaches is more amenable to learning than the other, we hope to have contributed to the development of the area by approaching it from the perspective of symbolic and sub-symbolic integration. (shrink)

Ramsey, Stich and Garon's recent paper Connectionism, Eliminativism, and the Future of Folk Psychology claims a certain style of connectionism to be the final nail in the coffin of folk psychology. I argue that their paper fails to show this, and that the style of connectionism they illustrate can in fact supplement, rather than compete with, the claims of a theory of cognition based in folk psychology's ontology. Ramsey, Stich and Garon's argument relies on the lack of easily identifiable symbols (...) inside the connectionist network they discuss, and they suggest that the existence of a system which behaves in a cognitively interesting way, but which cannot be explained by appeal to internal symbol processing, falsifies central assumptions of folk psychology. My claim is that this argument is flawed, and that the theorist need not discard folk psychology in order to accept that the network illustrated exhibits cognitively interesting behaviour, even if it is conceded that symbols cannot be readily identified within the network. (shrink)

Abduction is or subsumes a process of inference. It entertains possible hypotheses and it chooses hypotheses for further scrutiny. There is a large literature on various aspects of non-symbolic, subconscious abduction. There is also a very active research community working on the symbolic (logical) characterisation of abduction, which typically treats it as a form of hypothetico-deductive reasoning. In this paper we start to bridge the gap between the symbolic and sub-symbolic approaches to abduction. We are interested in benefiting from developments (...) made by each community. In particular, we are interested in the ability of non-symbolic systems (neural networks) to learn from experience using efficient algorithms and to perform massively parallel computations of alternative abductive explanations. At the same time, we would like to benefit from the rigour and semantic clarity of symbolic logic. We present two approaches to dealing with abduction in neural networks. One of them uses Connectionist Modal Logic and a translation of Horn clauses into modal clauses to come up with a neural network ensemble that computes abductive explanations in a top-down fashion. The other combines neural-symbolic systems and abductive logic programming and proposes a neural architecture which performs a more systematic, bottom-up computation of alternative abductive explanations. Both approaches employ standard neural network architectures which are already known to be highly effective in practical learning applications. Differently from previous work in the area, our aim is to promote the integration of reasoning and learning in a way that the neural network provides the machinery for cognitive computation, inductive learning and hypothetical reasoning, while logic provides the rigour and explanation capability to the systems, facilitating the interaction with the outside world. Although it is left as future work to determine whether the structure of one of the proposed approaches is more amenable to learning than the other, we hope to have contributed to the development of the area by approaching it from the perspective of symbolic and sub-symbolic integration. (shrink)

The paper presents a paradoxical feature of computational systems that suggests that computationalism cannot explain symbol grounding. If the mind is a digital computer, as computationalism claims, then it can be computing either over meaningful symbols or over meaningless symbols. If it is computing over meaningful symbols its functioning presupposes the existence of meaningful symbols in the system, i.e. it implies semantic nativism. If the mind is computing over meaningless symbols, no intentional cognitive processes are available prior to (...) symbol grounding. In this case, no symbol grounding could take place since any grounding presupposes intentional cognitive processes. So, whether computing in the mind is over meaningless or over meaningful symbols, computationalism implies semantic nativism. (shrink)

Much research on intelligent systems has concentrated on low level mechanisms or sub-systems of restricted functionality. We need to understand how to put all the pieces together in an *architecture* for a complete agent with its own mind, driven by its own desires. A mind is a self-modifying control system, with a hierarchy of levels of control, and a different hierarchy of levels of implementation. AI needs to explore alternative control architectures and their implications for human, animal, and (...) artificial minds. Only within the framework of a theory of actual and possible architectures can we solve old problems about the concept of mind and causal roles of desires, beliefs, intentions, etc. The high level ``virtual machine'' architecture is more useful for this than detailed mechanisms. E.g. the difference between connectionist and symbolic implementations is of relatively minor importance. A good theory provides both explanations and a framework for systematically generating concepts of possible states and processes. Lacking this, philosophers cannot provide good analyses of concepts, psychologists and biologists cannot specify what they are trying to explain or explain it, and psychotherapists and educationalists are left groping with ill-understood problems. The paper sketches some requirements for such architectures, and analyses an idea shared between engineers and philosophers: the concept of ``semantic information''. (shrink)

The emerging contemporary natural philosophy provides a common ground for the integrative view of the natural, the artificial, and the human-social knowledge and practices. Learning process is central for acquiring, maintaining, and managing knowledge, both theoretical and practical. This paper explores the relationships between the present advances in understanding of learning in the sciences of the artificial (deep learning, robotics), natural sciences (neuroscience, cognitive science, biology), and philosophy (philosophy of computing, philosophy of mind, natural philosophy). The question is, what at (...) this stage of the development the inspiration from nature, specifically its computational models such as info-computation through morphological computing, can contribute to machine learning and artificial intelligence, and how much on the other hand models and experiments in machine learning and robotics can motivate, justify, and inform research in computational cognitive science, neurosciences, and computing nature. We propose that one contribution can be understanding of the mechanisms of ‘learning to learn’, as a step towards deep learning with symbolic layer of computation/information processing in a framework linking connectionism with symbolism. As all natural systems possessing intelligence are cognitive systems, we describe the evolutionary arguments for the necessity of learning to learn for a system to reach human-level intelligence through evolution and development. The paper thus presents a contribution to the epistemology of the contemporary philosophy of nature. (shrink)

Some argue the common practice of inferring multiple processes or systems from a dissociation is flawed. One proposed solution is state-trace analysis, which involves plotting, across two or more conditions of interest, performance measured by either two dependent variables, or two conditions of the same dependent measure. The resulting analysis is considered to provide evidence that either a single process underlies performance or there is evidence for more than one process. This article reports simulations using the simple recurrent network (...) in which changes to the learning rate produced state-trace plots with multiple functions. We also report simulations using a single-layer error-correcting network that generate plots with a single function. We argue that the presence of different functions on a state-trace plot does not necessarily support a dual-system account, at least as typically defined ; it can also indicate variation in a single parameter within theories generally considered to be single-system accounts. (shrink)

Any description of the emergence and evolution of different types of meaning processes (semiosis, sensu C.S.Peirce) in living systems must be supported by a theoretical framework which makes it possible to understand the nature and dynamics of such processes. Here we propose that the emergence of semiosis of different kinds can be understood as resulting from fundamental interactions in a triadically-organized hierarchical process. To grasp these interactions, we develop a model grounded on Stanley Salthe's hierarchical structuralism. This model can (...) be applied to establish, in a general sense, a set of theoretical constraints for explaining the instantiation of different kinds of meaning processes (iconic, indexical, symbolic) in semiotic systems. We use it to model a semiotic process in the immune system, namely, B-cell activation, in order to offer insights into the heuristic role it can play in the development of explanations for specific semiotic processes. (shrink)

Connectionism was explicitly put forward as an alternative to classical cognitive science. The questions arise: how exactly does connectionism differ from classical cognitive science, and how is it potentially better? The classical “rules and representations” conception of cognition is that cognitive transitions are determined by exceptionless rules that apply to the syntactic structure of symbols. Many philosophers have seen connectionism as a basis for denying structured symbols. We, on the other hand, argue that cognition is too rich and flexible to (...) be simulable by the exceptionless representation-level rules that classicism requires. However, this very richness of cognition requires syntactically structured representations—what philosophers call a language of thought. The natural mathematical characterization of neural networks comes from the theory of dynamical systems. We propose that the mathematics of dynamical systems, not the mathematics of algorithms, holds the key to how cognitive structure and cognitive processes can be realized in the physical structure and processes of a network. (shrink)

In this article, I survey the integrated connectionist/symbolic (ICS) cognitive architecture in which higher cognition must be formally characterized on two levels of description. At the microlevel, parallel distributed processing (PDP) characterizes mental processing; this PDP system has special organization in virtue of which it can be characterized at the macrolevel as a kind of symbolic computational system. The symbolic system inherits certain properties from its PDP substrate; the symbolic functions computed constitute optimization of a well-formedness measure (...) called Harmony. The most important outgrowth of the ICS research program is optimality theory (Prince & Smolensky, 1993/2004), an optimization based grammatical theory that provides a formal theory of cross-linguistic typology. Linguistically, Harmony maximization corresponds to minimization of markedness or structural ill-formedness. Cognitive explanation in ICS requires the collaboration of symbolic and connectionist principles. ICS is developed in detail in Smolensky and Legendre (2006a); this article is a précis of and guide to those volumes. (shrink)

We examine two connectionist networks—a fractal learning neural network (FLNN) and a Simple Recurrent Network (SRN)—that are trained to process center-embedded symbol sequences. Previous work provides evidence that connectionist networks trained on infinite-state languages tend to form fractal encodings. Most such work focuses on simple counting recursion cases (e.g., anbn), which are not comparable to the complex recursive patterns seen in natural language syntax. Here, we consider exponential state growth cases (including mirror recursion), describe a new training (...) scheme that seems to facilitate learning, and note that the connectionist learning of these cases has a continuous metamorphosis property that looks very different from what is achievable with symbolic encodings. We identify a property—ragged progressive generalization—which helps make this difference clearer. We suggest two conclusions. First, the fractal analysis of these more complex learning cases reveals the possibility of comparing connectionist networks and symbolic models of grammatical structure in a principled way—this helps remove the black box character of connectionist networks and indicates how the theory they support is different from symbolic approaches. Second, the findings indicate the value of future, linked mathematical and empirical work on these models—something that is more possible now than it was 10 years ago. (shrink)

Economic value additions to knowledge and demand provide practical, embedded and extensible meaning to philosophizing cognitive systems. Evaluation of a cognitive system is an empirical matter. Thinking of science in terms of distributed cognition (interactionism) enlarges the domain of cognition. Anything that actually contributes to the specific quality of output of a cognitive system is part of the system in time and/or space. Cognitive science studies behaviour and knowledge structures of experts and categorized structures based on underlying structures. Knowledge (...) representation through understanding of ‘epistemic cultures’ is an evolutionary stage. But cognition goes beyond knowledge representation. Notwithstanding the importance of epistemology of phenomena, the practicability cum philosophical aspects of machine learning needs to be seen in dynamic behaviour in socio-economic-technical value additions if human machine interaction processes that are context specific are incorporated into strong artificial intelligent systems. Cognitive Science is also studied from both computational and biological angles. Evolution of interactive forms of reasoning through understanding of meta-language of computations or biological learning processes is possible. But the limitation of historical cultures predefines the role of interactive processes in user-networks beyond technology networks. Despite this limitation, inclusive development notions of a heterogeneous national society such as India or Europe can be tested and incorporated. (shrink)

In the late 1980s, there were many who heralded the emergence of connectionism as a new paradigm – one which would eventually displace the classically symbolic methods then dominant in AI and Cognitive Science. At present, there remain influential connectionists who continue to defend connectionism as a more realistic paradigm for modeling cognition, at all levels of abstraction, than the classical methods of AI. Not infrequently, one encounters arguments along these lines: given what we know about neurophysiology, it is just (...) not plausible to suppose that our brains are digital computers. Thus, they could not support a classical architecture. I argue here for a middle ground between connectionism and classicism. I assume, for argument's sake, that some form(s) of connectionism can provide reasonably approximate models – at least for lower-level cognitive processes. Given this assumption, I argue on theoretical and empirical grounds that most human mental skills must reside in separate connectionist modules or sub-networks. Ultimately, it is argued that the basic tenets of connectionism, in conjunction with the fact that humans often employ novel combinations of skill modules in rule following and problem solving, lead to the plausible conclusion that, in certain domains, high level cognition requires some form of classical architecture. During the course of argument, it emerges that only an architecture with classical structure could support the novel patterns of information flow and interaction that would exist among the relevant set of modules. Such a classical architecture might very well reside in the abstract levels of a hybrid system whose lower-level modules are purely connectionist. (shrink)

The emerging contemporary natural philosophy provides a common ground for the integrative view of the natural, the artificial, and the human-social knowledge and practices. Learning process is central for acquiring, maintaining, and managing knowledge, both theoretical and practical. This paper explores the relationships between the present advances in understanding of learning in the sciences of the artificial, natural sciences, and philosophy. The question is, what at this stage of the development the inspiration from nature, specifically its computational models such as (...) info-computation through morphological computing, can contribute to machine learning and artificial intelligence, and how much on the other hand models and experiments in machine learning and robotics can motivate, justify, and inform research in computational cognitive science, neurosciences, and computing nature. We propose that one contribution can be understanding of the mechanisms of ‘learning to learn’, as a step towards deep learning with symbolic layer of computation/information processing in a framework linking connectionism with symbolism. As all natural systems possessing intelligence are cognitive systems, we describe the evolutionary arguments for the necessity of learning to learn for a system to reach humanlevel intelligence through evolution and development. The paper thus presents a contribution to the epistemology of the contemporary philosophy of nature. (shrink)

In contemporary artificial intelligence, the challenge is making intricate connectionist systems—comprising millions of parameters—more comprehensible, defensible, and rationally grounded. Two prevailing methodologies address this complexity. The inaugural approach amalgamates symbolic methodologies with connectionist paradigms, culminating in a hybrid system. This strategy systematizes extensive parameters within a limited framework of formal, symbolic rules. Conversely, the latter strategy remains staunchly connectionist, eschewing hybridity. Instead of internal transparency, it fabricates an external, transparent proxy system. This ancillary system’s mandate is (...) elucidating the principal system’s decisions, essentially approximating its outcomes. Leveraging natural language processing as our analytical lens, this paper elucidates both methodologies: the hybrid method is underscored by the compositional vector semantics, whereas the purely connectionist method evolves as a derivative of neural semantic parsers. This discourse extols the merits of the purely connectionist approach for its inherent flexibility and for a pivotal delineation: segregating the explanatory apparatus from the operational core, thereby rendering artificial intelligence systems reminiscent of human cognition. (shrink)

Systems based on symbolic knowledge have performed extremely well in processing reason, yet, remain beset with problems of brittleness in many domains. Connectionist approaches do similarly well in emulating interactive domains, however, have struggled when modelling higher brain functions. Neither of these dichotomous approaches, however, have provided many inroads into the area of human reasoning that psychology and sociology refer to as the process of practice. This paper argues that the absence of a model for the process (...) of practise in current approaches is a significant contributor to brittleness. This paper will investigate how the process of practise relates to deeper forms of contextual representations of knowledge. While researchers and developers of knowledge based systems have often incorporated the notion of context they treat context as a static entity, neglecting many connectionists’ work in learning hidden and dynamic contexts. This paper argues that the omission of these higher forms of context is one of the fundamental problems in the application and interpretation of symbolic knowledge. Finally, these ideas for modelling context will lead to the reinterpretation of situation cognition which makes a significant step towards a philosophy of knowledge that could lead to the modelling of the process of practice. (shrink)

The aim of the paper is to present the underlying reason of the unsolved symbol grounding problem. The Church-Turing Thesis states that a physical problem, for which there is an algorithm of solution, can be solved by a Turing machine, but machine operations neglect the semantic relationship between symbols and their meaning. Symbols are objects that are manipulated on rules based on their shapes. The computations are independent of the context, mental states, emotions, or feelings. The symbol (...) class='Hi'>processing operations are interpreted by the machine in a way quite different from the cognitive processes. Cognitive activities of living organisms and computation differ from each other, because of the way they act in the real word. The result is the problem of mutual understanding of symbol grounding. (shrink)

Evidence that an animal is capable of some degree of symbolic, human language processing supports the argument that the animal's consciousness is to some degree human-like. In this paper, we reinterpret the findings of Savage- Rumbaugh et al. using the twin tools of Deacon's referential hierarchy and Systemic Functional Linguistics, with a view to providing further corroborative evidence for a Bonobo ape's symbolic processing abilities, and as a result to open a window into the consciousness of at least (...) one non-human primate. (shrink)

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely (...) on a dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP. (shrink)

The emerging contemporary natural philosophy provides a common ground for the integrative view of the natural, the artificial, and the human-social knowledge and practices. Learning process is central for acquiring, maintaining, and managing knowledge, both theoretical and practical. This paper explores the relationships between the present advances in understanding of learning in the sciences of the artificial, natural sciences, and philosophy. The question is, what at this stage of the development the inspiration from nature, specifically its computational models such as (...) info-computation through morphological computing, can contribute to machine learning and artificial intelligence, and how much on the other hand models and experiments in machine learning and robotics can motivate, justify, and inform research in computational cognitive science, neurosciences, and computing nature. We propose that one contribution can be understanding of the mechanisms of ‘learning to learn’, as a step towards deep learning with symbolic layer of computation/information processing in a framework linking connectionism with symbolism. As all natural systems possessing intelligence are cognitive systems, we describe the evolutionary arguments for the necessity of learning to learn for a system to reach humanlevel intelligence through evolution and development. The paper thus presents a contribution to the epistemology of the contemporary philosophy of nature. (shrink)

The article criticises the attempt to establish connectionism as an alternative theory of human cognitive architecture through the introduction of thesymbolic/subsymbolic distinction (Smolensky, 1988). The reasons for the introduction of this distinction are discussed and found to be unconvincing. It is shown that thebrittleness problem has been solved for a large class ofsymbolic learning systems, e.g. the class oftop-down induction of decision-trees (TDIDT) learning systems. Also, the process of articulating expert knowledge in rules seems quite practical for many (...) important domains, including common sense knowledge.The article discusses several experimental comparisons betweenTDIDT systems and artificial neural networks using the error backpropagation algorithm (ANNs usingBP). The properties of one of theTDIDT systemsID3 (Quinlan, 1986a) are examined in detail. It is argued that the differences in performance betweenANNs usingBP andTDIDT systems reflect slightly different inductive biases but are not systematic; these differences do not support the view that symbolic and subsymbolic systems are fundamentally incompatible. It is concluded, that thesymbolic/subsymbolic distinction is spurious. It cannot establish connectionism as an alternative cognitive architecture. (shrink)

Computation is central to the foundations of modern cognitive science, but its role is controversial. Questions about computation abound: What is it for a physical system to implement a computation? Is computation sufficient for thought? What is the role of computation in a theory of cognition? What is the relation between different sorts of computational theory, such as connectionism and symbolic computation? In this paper I develop a systematic framework that addresses all of these questions. Justifying the role of computation (...) requires analysis of implementation, the nexus between abstract computations and concrete physical systems. I give such an analysis, based on the idea that a system implements a computation if the causal structure of the system mirrors the formal structure of the computation. This account can be used to justify the central commitments of artificial intelligence and computational cognitive science: the thesis of computational sufficiency, which holds that the right kind of computational structure suffices for the possession of a mind, and the thesis of computational explanation, which holds that computation provides a general framework for the explanation of cognitive processes. The theses are consequences of the facts that (a) computation can specify general patterns of causal organization, and (b) mentality is an organizational invariant, rooted in such patterns. Along the way I answer various challenges to the computationalist position, such as those put forward by Searle. I close by advocating a kind of minimal computationalism, compatible with a very wide variety of empirical approaches to the mind. This allows computation to serve as a true foundation for cognitive science. (shrink)

Dual-mechanism models of language maintain a distinction between a lexicon and a computational system of linguistic rules. In his target article, Clahsen provides support for such a distinction, presenting evidence from German inflections. He argues for a structured lexicon, going beyond the strict lexicon versus rules dichotomy. We agree with the author in assuming a dual mechanism; however, we argue that a next step must be taken, going beyond the notion of the computational system as specific rules applying to a (...) linguistic domain. By assuming a richer lexicon, the computational system can be conceived as a more general binding process that applies to different linguistic levels: syntax, morphology, reading, and spelling. (shrink)

I consider the role of cognitive modeling in cognitive science. Modeling, and the computers that enable it, are central to the field, but the role of modeling is often misunderstood. Models are not intended to capture fully the processes they attempt to elucidate. Rather, they are explorations of ideas about the nature of cognitive processes. In these explorations, simplification is essential—through simplification, the implications of the central ideas become more transparent. This is not to say that simplification has no downsides; (...) it does, and these are discussed. I then consider several contemporary frameworks for cognitive modeling, stressing the idea that each framework is useful in its own particular ways. Increases in computer power (by a factor of about 4 million) since 1958 have enabled new modeling paradigms to emerge, but these also depend on new ways of thinking. Will new paradigms emerge again with the next 1,000‐fold increase? (shrink)

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly (...) elegant and powerful models of memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. (shrink)

The representation of knowledge in the law has basically followed a rule-based logical-symbolic paradigm. This paper aims to show how the modeling of legal knowledge can be re-examined using connectionist models, from the perspective of the theory of the dynamics of unstable systems and chaos. We begin by showing the nature of the paradigm shift from a rule-based approach to one based on dynamic structures and by discussing how this would translate into the field of theory of law. (...) In order to show the full potential of this new approach, we start from an experiment with NEUROLEX, in which a neural network was used to model a corpus of French Council of State decisions. We examine the implications of this experiment, especially those concerning the limits of the model used, and show that other connectionist models might correspond more adequately to the nature of legal knowledge. Finally, we propose another neural model which could show not only the rules which emerge from legal qualification (NEUROLEX's goal), but also the way in which a legal qualification process evolves from one concept to another. (shrink)

Computation is central to the foundations of modern cognitive science, but its role is controversial. Questions about computation abound: What is it for a physical system to implement a computation? Is computation sufficient for thought? What is the role of computation in a theory of cognition? What is the relation between different sorts of computational theory, such as connectionism and symbolic computation? In this paper I develop a systematic framework that addresses all of these questions. Justifying the role of computation (...) requires analysis of implementation, the nexus between abstract computations and concrete physical systems. I give such an analysis, based on the idea that a system implements a computation if the causal structure of the system mirrors the formal structure of the computation. This account can be used to justify the central commitments of artificial intelligence and computational cognitive science: the thesis of computational sufficiency, which holds that the right kind of computational structure suffices for the possession of a mind, and the thesis of computational explanation, which holds that computation provides a general framework for the explanation of cognitive processes. The theses are consequences of the facts that computation can specify general patterns of causal organization, and mentality is an organizational invariant, rooted in such patterns. Along the way I answer various challenges to the computationalist position, such as those put forward by Searle. I close by advocating a kind of minimal computationalism, compatible with a very wide variety of empirical approaches to the mind. This allows computation to serve as a true foundation for cognitive science. (shrink)

People often solve spatially presented cognitive problems more easily than their nonspatial counterparts. We explain this phenomenon by characterizing space as an inter-modality that provides common structure to different specific perceptual modalities. The usefulness of spatial structure for knowledge processing on different levels of granularity and for interaction between internal and external processes is described. Map representations are discussed as examples in which the usefulness of spatially organized symbols is particularly evident. External representations and processes can enhance internal representations (...) and processes effectively when the same structures and principles can be implicitly assumed. (shrink)

The organization of the complex mixed-mode oscillations generated, in a three-dimensional variable space, by an autocatalytic process formalized as a cubic monomial is analyzed. The generation of the temporal patterns is elucidated by complementary approaches dealing with the three-variable differential continuous system itself and with successive discrete applications modelling its first return map. The extent to which the underlying bifurcation structures could constitute a fingerprint of autocatalytic processes is discussed in connection with the modelling of biological systems.

Uses connectionism (neural networks) to extract the "gist" of a story in order to represent a context going forward for the disambiguation of incoming words as a text is processed.

Connectionism and the Mind provides a clear and balanced introduction to connectionist networks and explores theoretical and philosophical implications. Much of this discussion from the first edition has been updated, and three new chapters have been added on the relation of connectionism to recent work on dynamical systems theory, artificial life, and cognitive neuroscience. Read two of the sample chapters on line: Connectionism and the Dynamical Approach to Cognition: http://www.blackwellpublishing.com/pdf/bechtel.pdf Networks, Robots, and Artificial Life: http://www.blackwellpublishing.com/pdf/bechtel2.pdf.

There is a school of thought which links connectionist models of cognition to eliminativism-the thesis that the constructs of commonsense psychology do not exist. This way of construing the impact of connectionist modelling is, I argue, deeply mistaken and depends crucially on a shallow analysis of the notion of explanation. I argue that good, higher level descriptions may group together physically heterogenous mechanisms, and that the constructs of folk psychology may fulfil such a grouping function even if they (...) fail to pick out discrete states of individual processing. More speculatively, the paper goes on to suggest that the virtues which recommend such constructs from a theorist’s third person perspective may nonetheless have close analogues in individual processing. Various difficulties for what I term a pure distributed connectionism may be solved by systems which utilise both distributed and classical symbolic representations-the latter exhibiting the discreteness and semantic interpretability which the eliminativist requires to vindicate common-sense psychology. If human beings turned out to be such mixed systems, then the eliminativist claim would be doubly misguided. It would be false as a conditional since even if pure distributed connectionism were a complete and accurate formal model of individual processing, it would not follow that the other, higher level constructs were not accurate and essential tools for a different kind of explanation. And it would involve a false antecedent since in a mixed system the symbolic descriptions may indeed be incarnate in the system’s own individual processing. The paper is in four main sections. Section 1 attempts to systematise a certain received picture of the relative status and accuracy of various levels of description of a connectionist system. Section 2 then introduces a general model of explanations which aim to group systems into equivalence classes defined for various purposes. Each grouping requires a special vocabulary and the constructs of any given vocabulary are legitimate just insofar as the grouping is interesting and useful. Section 3 goes on to show that, relative to such a model of explanation, the constructs of both symbolic AI and commonsense psychology may have a legitimate role to play in giving psychological explanations. This role is not just that of a useful approximation. The paper ends with a speculative section in which the argument for the theoretical usefulness of such symbolic constructs is extended, in a very natural way, to the domain of individual processing. Here the cognizer, in the process of regulating, debugging and understanding her own representations, creates symbols to stand for sets of distributed activity patterns. The section points out some difficulties for a pure distributed approach which may be eased by the addition of such symbolic constructs and relates the speculations to the current debate over the ’correct’architecture of cognition. If the speculations are on target, this whole debate turns out to be fundamentally illposed. (shrink)

The introduction of connectionist or parallel distributed processing (PDP) systems to model cognitive functions has raised the question of the possible relations between these models and traditional information processing models which employ rules to manipulate representations. After presenting a brief account of PDP models and two ways in which they are commonly interpreted by those seeking to use them to explain cognitive functions, I present two ways one might relate these models to traditional information processing (...) models and so not totally repudiate the tradition of modelling cognition through systems of rules and representations. The proposal that seems most promising is that PDP-type structures might provide the underlying framework in which a rule and representation model might be implemented. To show how one might pursue such a strategy, I discuss recent research by Barsalou on the instability of concepts and show how that might be accounted for in a system whose microstructure had a PDP architecture. I also outline how adopting a multi-leveled view of the mind, where on one level the mind employed a PDP-type system and at another level constituted a rule processing system, would allow researchers to relocate some problems which seemed difficult to explain at one level, such as the capacity for concept learning, to another level where it could be handled in a straightforward manner. (shrink)

In the first section of the article, we examine some recent criticisms of the connectionist enterprise: first, that connectionist models are fundamentally behaviorist in nature (and, therefore, non-cognitive), and second that connectionist models are fundamentally associationist in nature (and, therefore, cognitively weak). We argue that, for a limited class of connectionist models (feed-forward, pattern-associator models), the first criticism is unavoidable. With respect to the second criticism, we propose that connectionist modelsare fundamentally associationist but that this (...) is appropriate for building models of human cognition. However, we do accept the point that there are cognitive capacities for which any purely associative model cannot provide a satisfactory account. The implication that we draw from is this is not that associationist models and mechanisms should be scrapped, but rather that they should be enhanced.In the next section of the article, we identify a set of connectionist approaches which are characterized by “active symbols” — recurrent circuits which are the basis of knowledge representation. We claim that such approaches avoid criticisms of behaviorism and are, in principle, capable of supporting full cognition. In the final section of the article, we speculate at some length about what we believe would be the characteristics of a fully realized active symbol system. This includes both potential problems and possible solutions (for example, mechanisms needed to control activity in a complex recurrent network) as well as the promise of such systems (in particular, the emergence of knowledge structures which would constitute genuine internal models). (shrink)

Recent work in the methodology of connectionist explanation has I'ocrrsccl on the notion of levels of explanation. Specific issucs in conncctionisrn hcrc intersect with rvider areas of debate in the philosophy of psychology and thc philosophy of science generally. The issues I raise in this chapter, then, are not unique to cognitive science; but they arise in new and important contexts when connectionism is taken seriously as a model of cognition. The general questions are the relation between levels and (...) the status of levels which have no obvious relation to others. In speaking of levels, what is the connection, if there is one, between explanation and ontology? Which, if any, conccpt of reduction is applicable to connectionist systems? What kind of legitinrtcy can the constructs of common sense psychology, or of that vclsion ol intentional realism represented by classical symbol-systems n I, hirvc irr ir full-scale connectionist theory of mind? (shrink)