Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statis- tics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement record- ing systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, (...) association areas in the brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The stor- age and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual expe- riences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a com- mon frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspec- tion (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and ab- stract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinato- rially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a per- ceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal sym- bol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored. (shrink)

Recent evidence from cognitive neuroscience suggests that certain cognitive processes employ perceptual representations. Inspired by this evidence, a few researchers have proposed that cognition is inherently perceptual. They have developed an innovative theoretical approach that rests on the notion of perceptual simulation and marshaled several general arguments supporting the centrality of perceptual representations to concepts. In this article, I identify a number of weaknesses in these arguments and defend a multiple semantic code approach that posits both (...) perceptual and non-perceptual representations. (shrink)

Perceptual symbol systems form a theoretically plausible alternative to amodal symbol systems. At this point it is unclear whether there is any truly diagnostic empirical evidence to decide between these systems. We outline some possible avenues of research in the domain of language comprehension that might yield such evidence. Language comprehension will be an important arena for tests of the two types of symbol systems.

Many recent cognitive studies reveal that human cognition is inherently perceptual, sharing systems with perception at both the conceptual and the neural levels. This paper introduces Barsalou's theory of perceptual symbols and explores its implications for philosophy of science. If perceptual symbols lie in the heart of conceptual processing, the process of attribute selection during concept representation, which is critical for defining similarity and thus for comparing taxonomies, can no longer be determined solely by background (...) beliefs. The analogous nature of perceptual symbols and the spatial nature of intraconceptual relations impose new constraints on attribute selection. These constraints help people with different background beliefs select compatible attributes, which constitute a common "platform" for taxonomy comparison. (shrink)

The perceptual symbol approach to knowledge representation combines structured frames and dynamic imagery. The perceptual symbol approach provides a good account of the representation of scientific models, of some types of naive theories held by children and adults, and of certain reconstructive memory phenomena. The ontological status of perceptual symbols is unclear and this form of representation does not succeed in accounting for all forms of human knowledge.

In his target article, Barsalou cites current work on emotion theory but does not explore its relevance for this project. The connection is worth pursuing, since there is a plausible case to be made that emotions form a distinct symbolic information processing system of their own. On some views, that system is argued to be perceptual: a direct connection with Barsalou's perceptual symbol systems theory. Also relevant is the hypothesis that there may be different modular subsystems within emotion (...) and the perennial tension between cognitive and perceptual theories of emotion. (shrink)

Barsalou proposes (sect. 4.1.6) that perceptual symbols play a role in language processing. Data from our laboratory document this role and suggest the sorts of constraints used by simulators during language comprehension.

Various defenses of amodal symbol systems are addressed, including amodal symbols in sensory-motor areas, the causal theory of concepts, supramodal concepts, latent semantic analysis, and abstracted amodal symbols. Various aspects of perceptual symbol systems are clarified and developed, including perception, features, simulators, category structure, frames, analogy, introspection, situated action, and development. Particular attention is given to abstract concepts, language, and computational mechanisms.

A metaphor can often create novel features in an object or a situation. This phenomenon has been particularly hard to account for using amodal symbol systems: although highlighting and downplaying can explain the shift of focus, it cannot explain how entirely new features can come about. We suggest here that the dynamism of perceptual symbol systems, particularly the notion of simulator, provides an elegant account of the creativity of metaphor. The elegance lies in the idea that the creation of (...) new features by a metaphor proceeds in the same way as the creation of regular features in a perceptual symbol. (shrink)

We agree with Barsalou's claim about the importance of perceptual symbols in a theory of abstract concepts. Yet we maintain that the richness of many abstract concepts arises from the metaphorical mapping of recurring patterns of perceptual, embodied experience to provide essential structure to these abstract ideas.

It is argued that three major attempts by Barsalou to specify what makes a perceptual symbol perceptual fail. It is suggested that one way to give such an account is to employ the symbols.

It is very tempting to try to reconcile perception and cognition perceptual symbol systems may be a good way to achieve this; but is there actually a perception-cognition continuum? We offer several arguments for and against the existence of such a continuum and in favor of the choice of perceptual symbol systems. One of these arguments is purely theoretical, some are based on PET-scan observations and others are based on research with handicapped subjects who have communication problems associated (...) with cerebral lesions. These arguments suggest that modal perceptual symbols do indeed exist and that perception and cognition might have a common neuronal basis; but perceptual and cognitive activities require the activation of different neuronal structures. (shrink)

Although Barsalou is right in identifying the importance of perceptual symbols as a means of carrying certain kinds of content, he is wrong in playing down the inferential resources available to amodal symbols. I argue that the case for perceptual symbol systems amounts to a false dichotomy and that it is feasible to help oneself to both kinds of content as extreme ends on a content continuum. The continuum thesis I advance argues for the inferential content (...) at one end and perceptual content at the other. In between the extremes, symbols might have aspects that are either perceptual or propositional-linguistic in character. I argue that this way of characterising the issue preserves the good sense of Barsalou's recognition of perceptual representations and yet avoids the tendency to minimise the gains won with symbolic representations vital to contemporary cognitive science. (shrink)

The main aim of this article is to present and defend a thesis according to which conceptual representations of some types of mental states are encoded in the same neural structures that underlie the first-personal experience of those states. To support this proposal here, I will put forth a novel account of the cognitive function played by ‘shared representations’ of emotions and bodily sensations, i.e. neural structures that are active when one experiences a mental state of a certain type as (...) well as when one observes someone else experiencing a state of the same type. I will argue that shared representations in fact constitute vehicles of certain mental state concepts (more precisely, concepts of specific types of emotions and somatosensory states). The main line of arguing for this will consist in showing that shared representations exhibit specific, ‘conceptual’ functional properties:(1) causal effect on forming metacognitive judgments, (2) cognitive penetrability, (3) diversity of input types. (shrink)

Barsalou's theory of perceptual symbol systems is considered from a metacognitive perspective. Two examples are discussed in terms of the proposed perceptual symbol theory. First, recent results in research on feeling-of-knowing judgement are used to argue for a representation of familiarity with input cues. This representation should support implicit memory. Second, the ability of maintaining a theory of other people's beliefs (theory of mind) is considered and it is suggested that a purely simulation-based view is insufficient to explain (...) the available evidence. Both examples characterize areas where Barsalou's theory would benefit from additional detail. (shrink)

Barsalou's theory rightly emphasizes the perceptual basis of cognition. However, the perceptual symbols that he proposes seem ill suited to carry the representational burden entailed by the architecture in which they function, given that Barsalou accepts the requirement for productivity. A more radical proposal is needed in which symbols are largely external to the cognizer and linked to internal states via perception.

People can be taught to manipulate symbols according to formal mathematical and logical rules. Cognitive scientists have traditionally viewed this capacity—the capacity for symbolic reasoning—as grounded in the ability to internally represent numbers, logical relationships, and mathematical rules in an abstract, amodal fashion. We present an alternative view, portraying symbolic reasoning as a special kind of embodied reasoning in which arithmetic and logical formulae, externally represented as notations, serve as targets for powerful perceptual and sensorimotor systems. Although symbolic (...) reasoning often conforms to abstract mathematical principles, it is typically implemented by perceptual and sensorimotor engagement with concrete environmental structures. (shrink)

According to the Perceptual Symbols Theory of cognition (Barsalou, 1999), modality‐specific simulations underlie the representation of concepts. A strong prediction of this view is that perceptual processing affects conceptual processing. In this study, participants performed a perceptual detection task and a conceptual property‐verification task in alternation. Responses on the property‐verification task were slower for those trials that were preceded by a perceptual trial in a different modality than for those that were preceded by a (...) class='Hi'>perceptual trial in the same modality. This finding of a modality‐switch effect across perceptual processing and conceptual processing supports the hypothesis that perceptual and conceptual representations are partially based on the same systems. (shrink)

This article proposes a number of models to examine through which mechanisms a population of autonomous agents could arrive at a repertoire of perceptually grounded categories that is sufficiently shared to allow successful communication. The models are inspired by the main approaches to human categorisation being discussed in the literature: nativism, empiricism, and culturalism. Colour is taken as a case study. Although we take no stance on which position is to be accepted as final truth with respect to human categorisation (...) and naming, we do point to theoretical constraints that make each position more or less likely and we make clear suggestions on what the best engineering solution would be. Specifically, we argue that the collective choice of a shared repertoire must integrate multiple constraints, including constraints coming from communication. Key Words: autonomous agents; colour categorisation; colour naming; connectionism; cultural evolution; genetic evolution; memes; origins of language; self-organisation; semiotic dynamics; symbol grounding. (shrink)

We are reminded of relevant stories, tales, or symbols from long-term memory when facing a novel problem our daily lives. Visual cues are 1 tool known to facilitate reminding. In 2 experiments, Chinese students, who had experienced a folk tale many years ago during childhood, were asked to solve an analogous problem. We tested the hypothesis that a visual cue can help bridge the gap between a novel problem and a source analogy experienced in the distant past. Different types (...) of cues proved to have distinct influences on components of analogical problem solving. This research suggests that analogical representational features of a story or problem can be stored in long-term memory and can be activated by visual cues. The implications of these findings for theories that point to perceptual or grounded, rather than decontextualized structures in knowledge representation are discussed. (shrink)

Whether computational algorithms such as latent semantic analysis (LSA) can both extract meaning from language and advance theories of human cognition has become a topic of debate in cognitive science, whereby accounts of symbolic cognition and embodied cognition are often contrasted. Albeit for different reasons, in both accounts the importance of statistical regularities in linguistic surface structure tends to be underestimated. The current article gives an overview of the symbolic and embodied cognition accounts and shows how meaning induction attributed to (...) a specific statistical process or to activation of embodied representations should be attributed to language itself. Specifically, the performance of LSA can be attributed to the linguistic surface structure, more than special characteristics of the algorithm, and embodiment findings attributed to perceptual simulations can be explained by distributional linguistic information. (shrink)

Symbols should be grounded, as has been argued before. But we insist that they should be grounded not only in subsymbolic activities, but also in the interaction between the agent and the world. The point is that concepts are not formed in isolation (from the world), in abstraction, or "objectively." They are formed in relation to the experience of agents, through their perceptual/motor apparatuses, in their world and linked to their goals and actions. This paper takes a detailed (...) look at this relatively old issue, with a new perspective, aided by our work of computational cognitive model development. To further our understanding, we also go back in time to link up with earlier philosophical theories related to this issue. The result is an account that extends from computational mechanisms to philosophical abstractions. (shrink)

This article identifies the structural and conceptual aspects of a visual construction often used in advertisements to establish a metaphoric or associative relation, that is, symmetric object alignment (SOA). It offers an account of the formal ingredients of SOA, which fall into two groups: object-constitutive factors (like size, shape, and color) and object-depictment factors (like perspective, orientation, and distance from viewing point). Both factors allow us to treat SOA as a visual rhetorical scheme. We also discuss how SOA relates to (...) conceptual structure, metaphoric or otherwise. Four types of relations are distinguished, based on the way in which perceptual and conceptual characteristics interact. Finally, we discuss the way in which these characteristics relate to the processing of such visual messages. One depictment factor will be focused on in particular: one-to-one versus one-to-many alignments. An attempt is made to connect these 2 patterns to the much-discussed topic in verbal metaphor theory, object comparison versus categorization. (shrink)

Cognitive scientists have a variety of approaches to studying cognition: experimental psychology, computer science, robotics, neuroscience, educational psychology, philosophy of mind, and psycholinguistics, to name but a few. In addition, they also differ in their approaches to cognition - some of them consider that the mind works basically like a computer, involving programs composed of abstract, amodal, and arbitrary symbols. Others claim that cognition is embodied - that is, symbols must be grounded on perceptual, motoric, and emotional (...) experience. The existence of such different approaches has consequences when dealing with practical issues such as understanding brain disorders, designing artificial intelligence programs and robots, improving psychotherapy, or designing instructional programs. The symbolist and embodiment camps seldom engage in any kind of debate to clarify their differences. This book is the first attempt to do so. It brings together a team of outstanding scientists, adopting symbolist and embodied viewpoints, in an attempt to understand how the mind works and the nature of linguistic meaning. As well as being interdisciplinary, all authors have made an attempt to find solutions to substantial issues beyond specific vocabularies and techniques. (shrink)

Abstract Since their inception, distributional models of semantics have been criticized as inadequate cognitive theories of human semantic learning and representation. A principal challenge is that the representations derived by distributional models are purely symbolic and are not grounded in perception and action; this challenge has led many to favor feature-based models of semantic representation. We argue that the amount of perceptual and other semantic information that can be learned from purely distributional statistics has been underappreciated. We compare the (...) representations of three feature-based and nine distributional models using a semantic clustering task. Several distributional models demonstrated semantic clustering comparable with clustering-based on feature-based representations. Furthermore, when trained on child-directed speech, the same distributional models perform as well as sensorimotor-based feature representations of children’s lexical semantic knowledge. These results suggest that, to a large extent, information relevant for extracting semantic categories is redundantly coded in perceptual and linguistic experience. Detailed analyses of the semantic clusters of the feature-based and distributional models also reveal that the models make use of complementary cues to semantic organization from the two data streams. Rather than conceptualizing feature-based and distributional models as competing theories, we argue that future focus should be on understanding the cognitive mechanisms humans use to integrate the two sources. (shrink)

Perspective, Symbol, and Symbolic Form: Concerning the Relationship between Cassirer and Panofsky During the last two decades of the twentieth century, there was a sudden surge of interest in Ernst Cassirer’s major work, The Philosophy of Symbolic Forms (1923–29), and Erwin Panofsky’s essay, ‘Perspective as Symbolic Form’ (1927), an interest that has continued uninterrupted to the present day. Particularly amongst art historians, however, a serious misunderstanding remains evident here – the confusing of ‘symbolic form’ with ‘symbol’. Cultural and perceptual (...) mediations, in which objects (and subjects) are only just in the process of forming, are carelessly turned into arbitrary, isolated objects of art history or pictorial history. Every work, in this view, is regarded as a ‘symbolic form’ to the extent that a representation of the world is ‘expressed’ in it. This article initially reviews Panofsky’s essay in order to establish the context in which the art historian uses the term ‘symbolic form’. His use of it is then compared with Cassirer’s original understanding of the term. A careful distinction is made between ‘symbol’, ‘symbolic pregnance’, and ‘symbolic form’, and this is followed by an analysis of scattered remarks in Cassirer’s writings, and particularly in his posthumous manuscripts and notes, on ‘art’ as symbolic form and on the spatial form that is prior to all perception and art production, as well as his call for a kind of art history that conceives of itself as a scholarly discipline. The article concludes with the recognition that Panofsky not only deliberately, but justifiably – that is, in the spirit of Cassirer, at least – transferred the expression ‘symbolic form’ to ‘perspective’. (shrink)

In an evolutionary perspective Louwerse elaborates the Symbol Interdependency Hypothesis (Louwerse, 2011), arguing that language has evolved such that it maps onto the perceptual system, allowing to bootstrap meaning also when grounding is limited. The author concludes that in principle the processing of abstract and concrete words is the same and that in both cases language users tend to rely anyway on indexical relationships that words entertain with other words.

The ArgumentErnst Cassirer's theory of language as a symbolic form, one of the richest and most insightful philosophies of language of the twentieth century, went virtually unnoticed in the mainstreams of modern linguistics. This was so for what seems to be a good metatheoretical reason: Cassirer insisted on the constitutive role of meaning in the explanation of linguistic phenomena, a position which was explicitly rejected by both American Structuralists and Chomskian Generativists. In the last decade, however, a new and promising (...) linguistic framework has emerged — the framework of lexical semantics — which seems to bear close theoretical resemblance to Cassirer's theory. In this paper, I show how the empirical results accumulated within the framework of lexical semantics serve to validate Cassirer's most fundamental philosophical insights, and suggest that Cassirer's philosophy helps position these empirical results in their appropriate epistemological context. I discuss the following fundamental points, which, for me, constitute the backbone of both Cassirer's philosophy and the theory of lexical semantics: natural language grammars constitute structural reflections of a deeply-rooted, highly structured level of semantic organization; the representational level of linguistic meaning, which is prior to experience in the Kantian sense, comprises a partial set of semantic notions, which language selects as centers of perceptual attention; this partial set is potentially different from the sets selected by other symbolic forms, such as myth, science, and art; and linguistic variability is to be explained in universalistic terms, thus allowing for specific patterns of variability within universally-constrained limits. (shrink)

Barsalou's elision of the personal and sub-personal levels tends to conceal the fact that he is, at best, providing the “specs” but not yet a model for his hypothesized perceptual symbols.

A dual-code model of number processing needs to take into account the difference between a number symbol and its meaning. The transition of automatic non-abstract number representations into intentional abstract representations could be conceptualized as a translation of perceptual asemantic representations of numerals into semantic representations of the associated magnitude information. The controversy about the nature of number representations should be thus related to theories on embodied grounding of symbols.

This is a complex work by the author of The Moral Nature of Man. First, it is an inventory of the perceptual world using as a tool an original distinction between noticing and observing. This leads to the establishment of a continuity between the conscious and the subconscious, and to the discernment of various meaning-giving levels of attention. Secondly, it is a review of opinion on sensation and perception in recent Anglo-American thought. Particular attention is given to the ideas (...) of Price, Ayer, and Austin. Thirdly, and most importantly, it contains a full theory of reality based on: 1) the resistance or centered action of dynamic systems; 2) the purposive action that meets this resistance; and 3) the emergence of qualia from the encounter of 1 and 2. The method is that of description of ordinary language conceptions of sensation and sensa, "things," interaction and causation, and symbol. From this analysis Garnett obtains some original conclusions about sense qualia and brain processes, body and mind, and, in general, about the physical and mental elements of the universe.—A. M. (shrink)

The well-known game of chess has traditionally been modeled in artificial intelligence studies by search engines with advanced pruning techniques. The models were thus centered on an inference engine manipulating passive symbols in the form of tokens. It is beyond doubt, however, that human players do not carry out such processes. Instead, chess masters instead carry out perceptual processes, carefully categorizing the chunks perceived in a position and gradually building complex dynamic structures to represent the subtle pressures embedded (...) in the positions. In this paper we will consider two hypotheses concerning the underlying subcognitive processes and architecture. In the first hypothesis, a multiple-leveled chess representational structure is presented, which includes distance graphs (with varying levels of quality) between pieces, piece mobilities, and abstract roles. These representational schemes seem to account for numerous characteristics of human player’s psychology. The second hypothesis concerns the extension of the architecture proposed in the Copycat project as central for modeling the emergent intuitive perception of a chess position. We provide a synthesis on how the postulated architecture models chess intuition as an emergent mixture of simultaneous distance estimations, chunk perceptions, abstract role awareness, and intention activations. This is an alternative model to the traditional AI approaches, focusing on the philosophy of active symbols. (shrink)

People often solve spatially presented cognitive problems more easily than their nonspatial counterparts. We explain this phenomenon by characterizing space as an inter-modality that provides common structure to different specific perceptual modalities. The usefulness of spatial structure for knowledge processing on different levels of granularity and for interaction between internal and external processes is described. Map representations are discussed as examples in which the usefulness of spatially organized symbols is particularly evident. External representations and processes can enhance internal (...) representations and processes effectively when the same structures and principles can be implicitly assumed. (shrink)

This is a complex work by the author of The Moral Nature of Man. First, it is an inventory of the perceptual world using as a tool an original distinction between noticing and observing. This leads to the establishment of a continuity between the conscious and the subconscious, and to the discernment of various meaning-giving levels of attention. Secondly, it is a review of opinion on sensation and perception in recent Anglo-American thought. Particular attention is given to the ideas (...) of Price, Ayer, and Austin. Thirdly, and most importantly, it contains a full theory of reality based on: 1) the resistance or centered action of dynamic systems; 2) the purposive action that meets this resistance; and 3) the emergence of qualia from the encounter of 1 and 2. The method is that of description of ordinary language conceptions of sensation and sensa, "things," interaction and causation, and symbol. From this analysis Garnett obtains some original conclusions about sense qualia and brain processes, body and mind, and, in general, about the physical and mental elements of the universe.—A. M. (shrink)

Locus equations are supposed to capture a perceptual invariant of place of articulation in consonants. Synthetic speech data show that human classification deviates systematically from the predictions of locus equations. The few studies that have contrasted predictions from competing theories yield mixed results, indicating that no current theory adequately characterizes the perceptual mapping from sound to phonetic symbol.

In Exploring Robotic Minds: Actions, Symbols, and Consciousness as Self-Organizing Dynamic Phenomena, Jun Tani sets out to answer an essential and tantalizing question: How do our minds work? By providing an overview of his "synthetic neurorobotics" project, Tani reveals how symbols and concepts that represent the world can emerge in a neurodynamic structure--iterative interactions between the top-down subjective view, which proactively acts on the world, and the bottom-up recognition of the resultant perceptual reality. He argues that nontrivial (...) problems of consciousness and free will could be addressed through structural understanding of such iterative, conflicting interactions between the top-down and the bottom-up pathways. A wide range of readers will enjoy this wonderful journey of the mind and will follow the author on interdisciplinary discussions that span neuroscience, dynamical systems theories, robotics, and phenomenology. The book also includes many figures, as well as a link to videos of Tani's exciting robotic experiments. (shrink)

Not long ago the standard view in cognitive science was that representations are symbols in an internal representational system or language of thought and that psychological processes are computations defined over such representations. This orthodoxy has been challenged by adherents of functional analysis and by connectionists. Functional analysis as practiced by Marr is consistent with an analysis of representation that grants primacy to a stands for conception of representation. Connectionism is also compatible with this notion of representation; when conjoined (...) with functional analysis, it provides a means of analyzing psychological systems in term of rules and representations without becoming committed to symbolism. Direct theorists, who rejected the orthodox symbolist conception of representation because it violated their strictures against cognitive mediational mechanisms, should find it possible to accept rules-and-representations and information-processing analyses of the mechanisms of information pickup couched in terms of functional analysis. (shrink)

I modify and generalize Carnap’s notion of frameworks as a way of unpacking Goodman’s metaphor of “making worlds with symbols”. My frameworks provide, metaphorically, a way of making worlds out of symbols in as much as all our framework-bound access to the world is through frameworks that always stand to be improved in accuracy, precision, and usually both. Such improvement is characterized in pragmatist terms.