Figure 1: A pr ototyp ical exa mple of a three-layer feed forward network, used by Plunkett and M archm an (1 991 ) to simulate learning the past-tense of En glish verbs. The inpu t units encode representations of the three phonemes of the present tense of the artificial words used in this simulation. Th e netwo rk is trained to produce a representation of the phonemes employed in the past tense form and the suffix (/d/, /ed/, or /t/) (...) used on regular verbs. To run the network, each input unit is assigned an activation value o f 0 or 1 , dep ending on whethe r the featu re is present or not. Eac h input unit is conne cted to each of the 30 hidden units by a we ighted conn ection and p rovid es an inp ut to each hidden unit equal to the product of the input unit's activation and the weight. Each hidd en unit's activation is then determined by summing ov er the va lues co ming fro m each inp ut unit to deter mine a netinput, and then applying a non-linear function (e.g., the logistic function 1/(1+e^netinput). Th is whole proced ure is. (shrink)

Robert Cummins [(1996) Representations, targets and attitudes, Cambridge, MA: Bradford/MIT, p. 1] has characterized the vexed problem of mental representation as "the topic in the philosophy of mind for some time now." This remark is something of an understatement. The same topic was central to the famous controversy between Nicolas Malebranche and Antoine Arnauld in the 17th century and remained central to the entire philosophical tradition of "ideas" in the writings of Locke, Berkeley, Hume, Reid and Kant. However, the scholarly, (...) exegetical literature has almost no overlap with that of contemporary cognitive science. I show that the recurrence of certain deep perplexities about the mind is a systematic and pervasive pattern arising not only throughout history, but also in a number of independent domains today such as debates over visual imagery, symbolic systems and others. Such historical and contemporary convergences suggest that the fundamental issues cannot arise essentially from the theoretical guise they take in any particular case. (shrink)

Various systems that learn are examined to show how content is carried in connections installed by a learning history. Agents do not explicitly use the content of such states in practical reasoning, yet the content plays an important role in explaining behavior, and the physical state carrying that content plays a role in causing behavior, given other occurrent beliefs and desires. This leads to an understanding of the environmental reasons which are the determinate content of these states, and leads to (...) a better grasp of how representational content can be carried by systems without an explicit representation. (shrink)

Does connectionism spell doom for folk psychology? I examine the proposal that cognitive representational states such as beliefs can play no role if connectionist models - - interpreted as radical new cognitive theories -- take hold and replace other cognitive theories. Though I accept that connectionist theories are radical theories that shed light on cognition, I reject the conclusion that neural networks do not represent. Indeed, I argue that neural networks may actually give us a better working notion of cognitive (...) representational states such as beliefs, and in so doing give us a better understanding of how these states might be instantiated in neural wetware. (shrink)

Does connectionism spell doom for folk psychology? I examine the proposal that cognitive representational states such as beliefs can play no role if connectionist models—interpreted as radical new cognitive theories—take hold and replace other cognitive theories. Though I accept that connectionist theories are radical theories that shed light on cognition, I reject the conclusion that neural networks do not represent. Indeed, I argue that neural networks may actually give us a better working notion of cognitive representational states such as beliefs, (...) and in so doing give us a better understanding of how these states might be instantiated in neural wetware. (shrink)

The relevance of Wolfgang K hler's psychoneural isomorphism principle to contemporary cognitive neuroscience is explored. K hler's approach to the mind—body problem is interpreted as a response to the foundational crisis of psychology at the beginning of the twentieth century. Some aspects of his isomorphism doctrine are discussed, with a view to reaching an interpretation that is both historically accurate and pertinent to issues currently debated in the philosophy of psychology. The principle was meant to be empirically verifiable. Accordingly, some (...) similarities between K hler's approach and current neural network modeling are pointed out, and it is shown that some recent trends in the neurosciences are broadly compatible with K hler's views on cortical functioning. Isomorphism is interpreted as a form of neuroreductionism constrained by bridging laws relating mental phenomena to macrosocopic parameters of neural function. While isomorphism is probably valid for perceptual phenomena, its applicability to higher mental processes remains doubtful. (shrink)

The question of whether cognition requires representations has engendered heated discussion during the last two decades. I shall argue that the question is, in all likelihood, a spurious one. There may or may not be a fact of the matter concerning whether a given item qualifies as a representation. However, even if there is, attempts to establish whether cognition requires representation have neither practical nor theoretical utility.

Recently, connectionist models have been developed that seem to exhibit structuresensitive cognitive capacities without executing a program. This paper examines one such model and argues that it does execute a program. The argument proceeds by showing that what is essential to running a program is preserving the functional structure of the program. It has generally been assumed that this can only be done by systems possessing a certain temporalcausal organization. However, counterfactualpreserving functional architecture can be instantiated in other ways, for (...) example geometrically, which are realizable by connectionist networks. (shrink)

In “Connectionism and the fats of folk psychology”, Forster and Saidel argue that the central claim of Ramsey, Stich and Garon (1991)—that distributed connectionist models are incompatible with the causal discreteness of folk psychology—is mistaken. To establish their claim, they offer an intriguing model which allegedly shows how distributed representations can function in a causally discrete manner. They also challenge our position regarding projectibility of folk psychology. In this essay, I offer a response to their account and show how their (...) model fails to demonstrate that our original argument was mistaken. While I will discuss several difficulties with their model, my primary criticism will be that the features of their model that are causally discrete are not truly distributed, while the features that are distributed are not really discrete. Concerning the issue of projectibility, I am more inclined to agree with Forster and Saidel and I offer a revised account of what we should have said originally. (shrink)

Along with the increasing popularity of connectionist language models has come a number of provocative suggestions about the challenge these models present to Chomsky's arguments for nativism. The aim of this paper is to assess these claims. We begin by reconstructing Chomsky's argument from the poverty of the stimulus and arguing that it is best understood as three related arguments, with increasingly strong conclusions. Next, we provide a brief introduction to connectionism and give a quick survey of recent efforts to (...) develop networks that model various aspects of human linguistic behavior. Finally, we explore the implications of this research for Chomsky's arguments. Our claim is that the relation between connectionism and Chomsky's views on innate knowledge is more complicated than many have assumed, and that even if these models enjoy considerable success the threat they pose for linguistic nativism is small. (shrink)

This paper analyzes the rapid and unexpected rise of deep learning within Artificial Intelligence and its applications. It tackles the possible reasons for this remarkable success, providing candidate paths towards a satisfactory explanation of why it works so well, at least in some domains. A historical account is given for the ups and downs, which have characterized neural networks research and its evolution from “shallow” to “deep” learning architectures. A precise account of “success” is given, in order to sieve out (...) aspects pertaining to marketing or sociology of research, and the remaining aspects seem to certify a genuine value of deep learning, calling for explanation. The alleged two main propelling factors for deep learning, namely computing hardware performance and neuroscience findings, are scrutinized, and evaluated as relevant but insufficient for a comprehensive explanation. We review various attempts that have been made to provide mathematical foundations able to justify the efficiency of deep learning, and we deem this is the most promising road to follow, even if the current achievements are too scattered and relevant for very limited classes of deep neural models. The authors’ take is that most of what can explain the very nature of why deep learning works at all and even very well across so many domains of application is still to be understood and further research, which addresses the theoretical foundation of artificial learning, is still very much needed. (shrink)

When cognitive scientists apply computational theory to the problem of phenomenal consciousness, as many of them have been doing recently, there are two fundamentally distinct approaches available. Either consciousness is to be explained in terms of the nature of the representational vehicles the brain deploys; or it is to be explained in terms of the computational processes defined over these vehicles. We call versions of these two approaches _vehicle_ and _process_ theories of consciousness, respectively. However, while there may be space (...) for vehicle theories of consciousness in cognitive science, they are relatively rare. This is because of the influence exerted, on the one hand, by a large body of research which purports to show that the explicit representation of information in the brain and conscious experience are _dissociable_, and on the other, by the _classical_ computational theory of mind – the theory that takes human cognition to be a species of symbol manipulation. But two recent developments in cognitive science combine to suggest that a reappraisal of this situation is in order. First, a number of theorists have recently been highly critical of the experimental methodologies employed in the dissociation studies – so critical, in fact, it’s no longer reasonable to assume that the dissociability of conscious experience and explicit representation has been adequately demonstrated. Second, classicism, as a theory of human cognition, is no longer as dominant in cognitive science as it once was. It now has a lively competitor in the form of _connectionism; _and connectionism, unlike classicism, does have the computational resources to support a robust vehicle theory of consciousness. In this paper we develop and defend this connectionist vehicle theory of consciousness. It takes the form of the following simple empirical hypothesis: _phenomenal experience consists in the explicit_ _representation of information in neurally realized PDP networks_.. (shrink)

One of the principal tasks Dennett sets himself in Consciousness Explained is to demolish the Cartesian theater model of phenomenal consciousness, which in its contemporary garb takes the form of Cartesian materialism: the idea that conscious experience is a process of presentation realized in the physical materials of the brain. The now standard response to Dennett is that, in focusing on Cartesian materialism, he attacks an impossibly naive account of consciousness held by no one currently working in cognitive science or (...) the philosophy of mind. Our response is quite different. We believe that, once properly formulated, Cartesian materialism is no straw man. Rather, it is an attractive hypothesis about the relationship between the computational architecture of the brain and phenomenal consciousness, and hence one that is worthy of further exploration. Consequently, our primary aim in this paper is to defend Cartesian materialism from Dennett’s assault. We do this by showing that Dennett’s argument against this position is founded on an implicit assumption (about the relationship between phenomenal experience and information coding in the brain), which while valid in the context of classical cognitive science, is not forced on connectionism. (shrink)

When it comes to applying computational theory to the problem of phenomenal consciousness, cognitive scientists appear to face a dilemma. The only strategy that seems to be available is one that explains consciousness in terms of special kinds of computational processes. But such theories, while they dominate the field, have counter-intuitive consequences; in particular, they force one to accept that phenomenal experience is composed of information processing effects. For cognitive scientists, therefore, it seems to come down to a choice between (...) a counter-intuitive theory or no theory at all. We offer a way out of this dilemma. We argue that the computational theory of mind doesn't force cognitive scientists to explain consciousness in terms of computational processes, as there is an alternative strategy available: one that focuses on the representational vehicles that encode information in the brain. This alternative approach to consciousness allows us to do justice to the standard intuitions about phenomenal experience, yet remain within the confines of cognitive science. (shrink)

O'Brien & Opie do not address the question of the psychotherapeutic role of unconscious representational states such as beliefs. A dilemma is proposed: if they accept the legitimacy of such states then they should modify what they say about dissociation, and if they do not, they owe us an account of why.

Connectionism: Debates on Psychological Explanation, Volume Two Edited by Cynthia Macdonald and Graham Macdonald Blackwell, 1995. Pp. xvii + 424. ISBN 0-631-19744-3. 50.00 (hbk). ISBN 0-631-19745-1 16.99 (pbk). New books on the philosophy of religion Divine Hiddenness and Human Reason By J.L. Schellenberg, Cornell University Press, 1993. Pp. 217. ISBN 0-8014-2792-4. $36.50 (hbk). Reason and the Heart By William J. Wainwright, Cornell University Press, 1995. Pp. 160. ISBN 0-8014-3139-5. $28.50 (hbk). The Rationality of Belief and the Plurality of Faith Edited (...) by Thomas D. Senor, Cornell University Press, 1995. Pp. 291. ISBN 0-8014-3127-1. $39.95 (hbk). (shrink)

Smolensky's (1995) proposal for a connectionist explanation of systematicity doesn't work.

The systematicity argument only challenges connectionism if systematicity is a general property of cognition. I examine this thesis in terms of properties of concepts. First, I propose that Evans's Generality Constraint only applies to attributions of belief. Then I defend a variety of conceptual pluralism, arguing that concepts share two fundamental properties related to centrality and belief-attribution, and contending that there are two kinds of concepts that differ in their compositional properties. Finally, I rely on Dual Systems Theory and on (...) differences between animal and human cognition to suggest a scenario of two processing systems that work on different kinds of concepts, with only one of them supporting full systematicity. I sketch a non-classical systematicity argument that rules out classicism as the basis of one of those systems given that it would wrongly entail that both systems are fully systematic. (shrink)

O'Brien & Opie's (O&O's) theory demands a view of unconscious processing that is incompatible with virtually all current PDP models of neural activity. Relative to the alternatives, the theory is closer to an AI than a parallel distributed processing (PDP) perspective, and its treatment of phenomenology is ad hoc. It raises at least one important question: Could features of network relaxation be the “switch” that turns an unconscious into a conscious network?

The rapid development of connectionist models in computer science and of powerful computational tools in neuroscience has encouraged eliminativist materialist philosophers to propose specific alternatives to traditional mentalistic theories of mind. One of the problems associated with such a move is that elimination of the mental would seem to remove access to ideas like truth as the foundations of normative epistemology. Thus, a successful elimination of propositional or sentential theories of mind must not only replace them for purposes of our (...) psychology, it must also replace them for purposes of the evaluation of our theories and explanations, psychological and otherwise. This paper briefly reviews eliminativist arguments for doubting the correctness of sentential accounts of explanation, understanding, and normative evaluation. It then considers Paul Churchland's (1989) proposed alternative norms, which are framed neurocomputationally. The alternative is found wanting in several specific ways. The arguments for eliminating propositionally-based norms are then re-examined and it is suggested that the need for wholesale elimination is overstated. A clear gap in the traditional epistemological story is identified, however, and a more modest set of norms is proposed as a way of filling this gap, rather than as a way of entirely replacing the traditional framework. (shrink)

In this paper we examine the question of whether complexity-like explanations can be applied to the psychology of individuals, and its implications for the scope of complexity explanations of social phenomena. We start by outlining two representational-cum-computational models of the mind—a symbolic model and a networks or connectionist one—and their pros and cons. Based on this we then outline a radical, non-representational and non-computational alternative model that has been gaining ground recently, and which has significant affinities with complexity explanations in (...) social science. Deploying neo-Kantian considerations, we then argue that due to the discursivity, or conceptual dimension of our cognitive system, the radical alternative must be incorrect insofar as humans are concerned. Indeed, human psychology must involve, at least partly, a representational understanding of the sort provided by the symbolic model. Relatedly, we show how the discursiviry of human cognition complicates our psychology and makes it difficult to account for. Finally, we briefly address the question of how the complicated nature of individual psychology, implied by human discursivity, may affect complexity explanations of social behavior. (shrink)

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of (...) memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. (shrink)

Ramsey, Stick and Garon (1991) argue that if the correct theory of mind is some parallel distributed processing theory, then folk psychology must be false. Their idea is that if the nodes and connections that encode one representation are causally active then all representations encoded by the same set of nodes and connections are also causally active. We present a clear, and concrete, counterexample to RSG's argument. In conclusion, we suggest that folk psychology and connectionism are best understood as complementary (...) theories. Each has different limitations, yet each will co-evolve with the other in an overlapping domain of 'normal' psychology. (shrink)

This paper examines whether a classical model could be translated into a PDP network using a standard connectionist training technique called extra output learning. In Study 1, standard machine learning techniques were used to create a decision tree that could be used to classify 8124 different mushrooms as being edible or poisonous on the basis of 21 different Features (Schlimmer, 1987). In Study 2, extra output learning was used to insert this decision tree into a PDP network being trained on (...) the identical problem. An interpretation of the trained network revealed a perfect mapping from its internal structure to the decision tree, representing a precise translation of the classical theory to the connectionist model. In Study 3, a second network was trained on the mushroom problem without using extra output learning. An interpretation of this second network revealed a different algorithm for solving the mushroom problem, demonstrating that the Study 2 network was indeed a proper theory translation. (shrink)

There is widespread belief that connectionist networks are dramatically different from classical or symbolic models. However, connectionists rarely test this belief by interpreting the internal structure of their nets. A new approach to interpreting networks was recently introduced by Berkeley et al. (1995). The current paper examines two implications of applying this method: (1) that the internal structure of a connectionist network can have a very classical appearance, and (2) that this interpretation can provide a cognitive theory that cannot be (...) dismissed as a mere implementation. (shrink)

Connectionism provides hope for unifying work in neuroscience, computer science, and cognitive psychology. This promise has met with some resistance from Classical Computionalists, which may have inspired Connectionists to retaliate with bold, inflationary claims on behalf of Connectionist models. This paper demonstrates, by examining three intimately connected issues, that these inflationary claims made on behalf of Connectionism are wrong. This should not be construed as an attack on Connectionism, however, since the inflated claims made on its behalf have the look (...) of cures for which there are no ailments. There is nothing wrong with Connectionism for its failure to solve illusory problems. (shrink)