Many kinds of creativity result from combination of mental representations. This paper provides a computational account of how creative thinking can arise from combining neural patterns into ones that are potentially novel and useful. We defend the hypothesis that such combinations arise from mechanisms that bind together neural activity by a process of convolution, a mathematical operation that interweaves structures. We describe computer simulations that show the feasibility of using convolution to produce emergent patterns of neural activity that can support (...) cognitive and emotional processes underlying human creativity. (shrink)

Darwin claimed that human and animal minds differ in degree but not in kind, and that ethical principles such as the Golden Rule are just an extension of thinking found in animals. Both claims are false. The best way to distinguish differences in degree from differences in kind is by identifying mechanisms that have emergent properties. Recursive thinking is an emergent capability found in humans but not in other animals. The Golden Rule and some other ethical principles such as Kant’s (...) categorical imperative require recursion, so they constitute ethical thinking that is restricted to humans. Changes in kind have tipping points resulting from mechanisms with emergent properties. (shrink)

Mental representations have continuous as well as discrete, combinatorial properties. For example, while predominantly discrete, phonological representations also vary continuously; this is reflected by gradient effects in instrumental studies of speech production. Can an integrated theoretical framework address both aspects of structure? The framework we introduce here, Gradient Symbol Processing, characterizes the emergence of grammatical macrostructure from the Parallel Distributed Processing microstructure (McClelland, Rumelhart, & The PDP Research Group, 1986) of language processing. The mental representations that emerge, Distributed Symbol Systems, (...) have both combinatorial and gradient structure. They are processed through Subsymbolic Optimization–Quantization, in which an optimization process favoring representations that satisfy well-formedness constraints operates in parallel with a distributed quantization process favoring discrete symbolic structures. We apply a particular instantiation of this framework, λ-Diffusion Theory, to phonological production. Simulations of the resulting model suggest that Gradient Symbol Processing offers a way to unify accounts of grammatical competence with both discrete and continuous patterns in language performance. (shrink)

In this article, I survey the integrated connectionist/symbolic (ICS) cognitive architecture in which higher cognition must be formally characterized on two levels of description. At the microlevel, parallel distributed processing (PDP) characterizes mental processing; this PDP system has special organization in virtue of which it can be characterized at the macrolevel as a kind of symbolic computational system. The symbolic system inherits certain properties from its PDP substrate; the symbolic functions computed constitute optimization of a well-formedness measure called Harmony. The (...) most important outgrowth of the ICS research program is optimality theory (Prince & Smolensky, 1993/2004), an optimization based grammatical theory that provides a formal theory of cross-linguistic typology. Linguistically, Harmony maximization corresponds to minimization of markedness or structural ill-formedness. Cognitive explanation in ICS requires the collaboration of symbolic and connectionist principles. ICS is developed in detail in Smolensky and Legendre (2006a); this article is a précis of and guide to those volumes. (shrink)

Inductive reasoning is a fundamental and complex aspect of human intelligence. In particular, how do subjects, given a set of particular examples, generate general descriptions of the rules governing that set? We present a biologically plausible method for accomplishing this task and implement it in a spiking neuron model. We demonstrate the success of this model by applying it to the problem domain of Raven's Progressive Matrices, a widely used tool in the field of intelligence testing. The model is able (...) to generate the rules necessary to correctly solve Raven's items, as well as recreate many of the experimental effects observed in human subjects. (shrink)

Classical (Bayesian) probability (CP) theory has led to an influential research tradition for modeling cognitive processes. Cognitive scientists have been trained to work with CP principles for so long that it is hard even to imagine alternative ways to formalize probabilities. However, in physics, quantum probability (QP) theory has been the dominant probabilistic approach for nearly 100 years. Could QP theory provide us with any advantages in cognitive modeling as well? Note first that both CP and QP theory share the (...) fundamental assumption that it is possible to model cognition on the basis of formal, probabilistic principles. But why consider a QP approach? The answers are that (1) there are many well-established empirical findings (e.g., from the influential Tversky, Kahneman research tradition) that are hard to reconcile with CP principles; and (2) these same findings have natural and straightforward explanations with quantum principles. In QP theory, probabilistic assessment is often strongly context- and order-dependent, individual states can be superposition states (that are impossible to associate with specific values), and composite systems can be entangled (they cannot be decomposed into their subsystems). All these characteristics appear perplexing from a classical perspective. However, our thesis is that they provide a more accurate and powerful account of certain cognitive processes. We first introduce QP theory and illustrate its application with psychological examples. We then review empirical findings that motivate the use of quantum theory in cognitive theory, but also discuss ways in which QP and CP theories converge. Finally, we consider the implications of a QP theory approach to cognition for human rationality. (shrink)

By pointing to deep philosophical confusions endemic to cognitive science, Wittgenstein might seem an enemy of computational approaches. We agree that while Wittgenstein would reject the classicist’s symbols and rules approach, his observations align well with connectionist or neural network approaches. While many connectionisms that dominated the later twentieth century could fall prey to criticisms of biological, pedagogical, and linguistic implausibility, current connectionist approaches can resolve those problems in a Wittgenstein-friendly manner. We present the basics of a Vector Symbolic Architecture (...) formalism, inspired by Smolensky, and indicate how high-dimensional vectors can operate in a context-sensitive and object-independent manner in biologically plausible time scales, reflecting Wittgenstein’s notions of language-games and family resemblance; we show how “soft” symbols for such a formalism can be formed with plausible learning cycles using Sparse Distributed Memory, resolving disputes surrounding Wittgenstein’s private language argument; and show how connectionist networks can extrapolate meaningful patterns to solve problems, providing “ways to go on” without explicit rules, which indicates linguistic plausibility. Connectionism thus provides a systematicity and productivity that is more than a mere implementation of a classical approach, and provides Wittgenstein-friendly and Wittgenstein-illuminating models of mind and language for cognitive science. (shrink)

According to Thagard and Stewart :1–33, 2011), creativity results from the combination of neural representations, and combination results from convolution, an operation on vectors defined in the holographic reduced representation framework. They use these ideas to understand creativity as it occurs in many domains, and in particular in science. We argue that, because of its algebraic properties, convolution alone is ill-suited to the role proposed by Thagard and Stewart. The semantic pointer concept allows us to see how we can apply (...) the full range of HRR operations while keeping the modal representations so central to Thagard and Stewart’s theory. By adding another combination operation and using semantic pointers as the combinatorial basis, this modified version overcomes the limitations of the original theory and perhaps helps us explain aspects of creativity not covered by the original theory. While a priori reasons cast doubts on the use of HRR operations with modal representations :5039–5054, 1987) such as semantic pointers, recent models point in the other direction, allowing us to be optimistic about the success of the revised version. (shrink)

We reformulate minimalist grammars as partial functions on term algebras for strings and trees. Using filler/role bindings and tensor product representations, we construct homomorphisms for these data structures into geometric vector spaces. We prove that the structure-building functions as well as simple processors for minimalist languages can be realized by piecewise linear operators in representation space. We also propose harmony, i.e. the distance of an intermediate processing step from the final well-formed state in representation space, as a measure of processing (...) complexity. Finally, we illustrate our findings by means of two particular arithmetic and fractal representations. (shrink)

Deep appreciation of the relevance of emotion to epistemology requires a rich account of how emotional mental states such as happiness, sadness and desire interact with cognitive states such as belief and doubt. Analytic philosophy since Gottlob Frege and Bertrand Russell has assumed that such mental states are propositional attitudes, which are relations between a self and a proposition, an abstract entity constituting the meaning of a sentence. This chapter shows the explanatory defects of the doctrine of propositional attitudes, and (...) proposes instead that beliefs, desires, and emotions should be construed naturalistically using current understanding of brain mechanisms. Mental states are patterns of neural activity, not relations between dubious entities such as selves and propositions. From this perspective, it becomes easy to see how cognition and emotion are intertwined, and hence how emotions can be integral to epistemology. I begin by reviewing some of the ways in which emotions are relevant to epistemology: as frequent contributors to the growth of knowledge, as sometime impediments to knowledge acquisition, and as components of knowledge about persons and morality. I then argue that propositional attitudes do not exist, because the selves and the propositions that they purportedly relate do not exist. Thus the doctrine of propositional attitudes is as useless for epistemology as it is for explaining human action. I argue for an alternative construal of mental states as patterns of neural activity, and July 23, 2007 describe how it is possible to give a theoretically rich and empirically supported account of the neurophysiological interconnections of cognition and emotion. Finally, I discuss the epistemological significance of this naturalistic, materialist reconstrual of cognitions and emotions. (shrink)

According to Millian Descriptivism, while the semantic content of a linguistically simple proper name is just its referent, we often use sentences containing such expressions “to make assertions…that are, in part, descriptive” (Soames 2008). Against this view, I show, following Ted Sider and David Braun (2006), that simple sentences containing names are never used to assert descriptively enriched propositions. In addition, I offer a diagnosis as to where the argument for Millian Descriptivism goes wrong. Once we appreciate the distinctive way (...) in which this account fails, we can better appreciate the very modest role that associated descriptive information plays in the pragmatics of proper names. (shrink)