Previous studies have shown that object properties are processed faster when they follow properties from the same perceptual modality than properties from different modalities. These findings suggest that language activates sensorimotor processes, which, according to those studies, can only be explained by a modal account of cognition. The current paper shows how a statistical linguistic approach of word co-occurrences can also reliably predict the category of perceptual modality a word belongs to (auditory, olfactory–gustatory, visual–haptic), even though the statistical (...) linguistic approach is less precise than the modal approach (auditory, gustatory, haptic, olfactory, visual). Moreover, the statistical linguistic approach is compared with the modal embodied approach in an experiment in which participants verify properties that share or shift modalities. Response times suggest that fast responses can best be explained by the linguistic account, whereas slower responses can best be explained by the embodied account. These results provide further evidence for the theory that conceptual processing is both linguistic and embodied, whereby less precise linguistic processes precede precise simulation processes. (shrink)

“Image metaphor” is defined in Conceptual Metaphor Theory as a mapping of visual structure from one entity onto another based on the mental images they evoke. It is considered an exceptional, one-off phenomenon that can be distinguished clearly from prototypical conceptual metaphors. However, according to Perceptual Simulation Theory, all language, both literal and nonliteral, is understood partially by simulating in our minds what it would be like to actually perceive the things that are being described, which suggests that (...) visualization is not unique to image metaphor. As I argue in this article, the mental imagery invited by image metaphors is nevertheless likely to be particularly vivid, detailed, and complex, as well as involving more conscious and deliberate thought processes. Since it is hard to consider visualization in abstract terms, pictorial equivalents of image metaphors are used to support my claims. (shrink)

How veridical is perception? Rather than representing objects as they actually exist in the world, might perception instead represent objects only in terms of the utility they offer to an observer? Previous work employed evolutionary modeling to show that under certain assumptions, natural selection favors such “strict‐interface” perceptual systems. This view has fueled considerable debate, but we think that discussions so far have failed to consider the implications of two critical aspects of perception. First, while existing models have explored (...) single utility functions, perception will often serve multiple largely independent goals. (Sometimes when looking at a stick you want to know how appropriate it would be as kindling for a campfire, and other times you want to know how appropriate it would be as a weapon for self‐defense.) Second, perception often operates in an inflexible, automatic manner—proving “impenetrable” to shifting higher‐level goals. (When your goal shifts from “burning” to “fighting,” your visual experience does not dramatically transform.) These two points have important implications for the veridicality of perception. In particular, as the need for flexible goals increases, inflexible perceptual systems must become more veridical. We support this position by providing evidence from evolutionary simulations that as the number of independent utility functions increases, the distinction between “interface” and “veridical” perceptual systems dissolves. Although natural selection evaluates perceptual systems only on their fitness, the most fit perceptual systems may nevertheless represent the world as it is. (shrink)

Empirical evidence suggests that perceptual‐motor simulations are often constitutively involved in language comprehension. Call this “the simulation view of language comprehension”. This article applies the simulation view to illuminate the much‐discussed phenomenon of copredication, where a noun permits multiple predications which seem to select different senses of the noun simultaneously. On the proposed account, the (in)felicitousness of a copredicational sentence is closely associated with the perceptual simulations that the language user deploys in comprehending the sentence.

Recent evidence from cognitive neuroscience suggests that certain cognitive processes employ perceptual representations. Inspired by this evidence, a few researchers have proposed that cognition is inherently perceptual. They have developed an innovative theoretical approach that rests on the notion of perceptual simulation and marshaled several general arguments supporting the centrality of perceptual representations to concepts. In this article, I identify a number of weaknesses in these arguments and defend a multiple semantic code approach that posits (...) both perceptual and non-perceptual representations. (shrink)

Classically, the question of recognizing another subject is posed unilaterally, in terms of the observed behaviour of the other entity. Here, we propose an alternative, based on the emergent patterns of activity resulting from the interaction of both partners. We employ a minimalist device which forces the subjects to externalize their perceptual activity as trajectories which can be observed and recorded; the results show that subjects do identify the situation of perceptual crossing with their partner. The interpretation of (...) the results is guided by comparable evolutionary robotics simulations. There are two components to subjects’ recognition capacities: distinguishing mobile and fixed entities; and behaving so as to interact with their partner rather than with a mobile lure. The “Other” is characterized by the feature that there is sufficient regularity in the interactions to encourage the formation of anticipations; but sufficient indetermination that the actual behaviour is consistently surprising. Keywords: Recognition of other; perceptual crossing; evolutionary robotics. (shrink)

The literature contains a disconnect between accounts of how humans learn lexical semantic representations for words. Theories generally propose that lexical semantics are learned either through perceptual experience or through exposure to regularities in language. We propose here a model to integrate these two information sources. Specifically, the model uses the global structure of memory to exploit the redundancy between language and perception in order to generate inferred perceptual representations for words with which the model has no (...) class='Hi'>perceptual experience. We test the model on a variety of different datasets from grounded cognition experiments and demonstrate that this diverse set of results can be explained as perceptual simulation (cf. Barsalou, Simmons, Barbey, & Wilson, 2003) within a global memory model. (shrink)

Recent application of theories of embodied or grounded cognition to the recognition and interpretation of facial expression of emotion has led to an explosion of research in psychology and the neurosciences. However, despite the accelerating number of reported findings, it remains unclear how the many component processes of emotion and their neural mechanisms actually support embodied simulation. Equally unclear is what triggers the use of embodied simulation versus perceptual or conceptual strategies in determining meaning. The present article (...) integrates behavioral research from social psychology with recent research in neurosciences in order to provide coherence to the extant and future research on this topic. The roles of several of the brain's reward systems, and the amygdala, somatosensory cortices, and motor centers are examined. These are then linked to behavioral and brain research on facial mimicry and eye gaze. Articulation of the mediators and moderators of facial mimicry and gaze are particularly useful in guiding interpretation of relevant findings from neurosciences. Finally, a model of the processing of the smile, the most complex of the facial expressions, is presented as a means to illustrate how to advance the application of theories of embodied cognition in the study of facial expression of emotion. (shrink)

This paper describes an operational computer simulation of visual mental imagery in humans. The structure of the simulation was motivated by results of experiments on how people represent information in, and access information from, visual images. The simulation includes a “surface representation,” which is spatial and quasi‐pictorial, and an underlying “deep representation,” which contains “perceptual” information encoding appearance plus “propositional” information describing facts about an object. The simulation embodies a theory of how surface images are (...) generated from deep representations, and how surface images are processed when one accesses information embedded in them. The simulation also offers an account of various sorts of imagery transformations. (shrink)

Embodied simulation and the epistemic motivation to search for the of other people's behaviors are not necessary for specific and functional responding to, and hence processing of, human facial expressions. Rather, facial expression processing can be achieved through lower-cognitive, heuristical perceptual processing and expression of prototypical morphological musculature movement patterns that communicate discrete trustworthiness and capacity cues to conspecifics.

This article criticises David Chalmers’ ‘Reality+’ by interrogating its distinction of virtual reality (VR) from 2D, non-VR video games, a distinction made on the grounds that VR is immersive and these types of video games are not because immersion is a distinct characteristic of 3D perceptually represented VR. Building on the Balcerak Jacksons’ account of ‘representational immersion’, which they acknowledge has ‘perceptual’ and ‘non-perceptual’ elements, I develop an account of ‘non-perceptual representational immersion’ and use it to critique (...) Chalmers’ treatment of VR, immersion, and video games, indirectly problematizing his central ontological claim that ‘virtual reality is genuine reality’ and the reflections he makes about future society which rest upon it. I argue that video games are rich in non-perceptual representational immersion, which includes the representation of non-perceptual information about the structure and behaviour of a virtual world and its systems, the non-perceptual cognitive states that are generated in the player by the virtual world, and the immersion which comes from the constitutively cognitive nature of the player’s engagement with the virtual world, a cognitive engagement construed in an enactive sense. These are just as crucial to the phenomenon of digital immersion as perception. Immersion generally is a much more complicated phenomenon than Chalmers allows for, and it is better conceived as taking place on a spectrum which includes both 3D and 2D experiences of virtual worlds, not as hinging on 3D perceptual representation. I use ‘immersive simulation’ games as a case study. (shrink)

According to the Perceptual Symbols Theory of cognition (Barsalou, 1999), modality‐specific simulations underlie the representation of concepts. A strong prediction of this view is that perceptual processing affects conceptual processing. In this study, participants performed a perceptual detection task and a conceptual property‐verification task in alternation. Responses on the property‐verification task were slower for those trials that were preceded by a perceptual trial in a different modality than for those that were preceded by a perceptual (...) trial in the same modality. This finding of a modality‐switch effect across perceptual processing and conceptual processing supports the hypothesis that perceptual and conceptual representations are partially based on the same systems. (shrink)

Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statis- tics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement record- ing systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, (...) association areas in the brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The stor- age and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual expe- riences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a com- mon frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspec- tion (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and ab- stract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinato- rially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a per- ceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal sym- bol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored. (shrink)

In an evolutionary perspective Louwerse elaborates the Symbol Interdependency Hypothesis (Louwerse, 2011), arguing that language has evolved such that it maps onto the perceptual system, allowing to bootstrap meaning also when grounding is limited. The author concludes that in principle the processing of abstract and concrete words is the same and that in both cases language users tend to rely anyway on indexical relationships that words entertain with other words.

Interest in thought experiments (TEs) derives from the paradox: “How can findings that carry conviction result from a new experiment conducted entirely within the head?” Historical studies have established the importance of TEs in science but have proposed disparate hypotheses concerning the source of knowledge in TEs, ranging from empiricist to rationalist accounts. This article analyzes TEs in think‐aloud protocols of scientifically trained experts to examine more fine‐grained information about their use. Some TEs appear powerful enough to discredit an existing (...) theory—a disconfirmatory purpose. In addition, confirmatory and generative purposes were identified for other TEs. One can also use details in transcript data, including imagery reports and gestures, to provide evidence for a central role played by imagistic simulations in many TEs, and to suggest that these simulations can generate new knowledge using several sources, including the “extended application” of perceptual motor schemas, implicit prior knowledge, and spatial reasoning operations, in contrast to formal arguments. These sources suggest what it means for TEs to be grounded in embodied processes that can begin to explain the paradox above. This leads to a rationalistic view of TEs as using productive internal reasoning, but the view also acknowledges the historical role that experience with the world can play in forming certain schemas used in TEs. Understanding such processes could help provide a foundation for developing a larger model of scientific investigation processes grounded on imagistic simulation (Clement, 2008). (shrink)

Barsalou's interesting model might benefit from defining simulation and clarifying the implications of prior critiques for simulations (and not just for perceptual symbols). Contrary to claims, simulators (or frames) appear, in the limit, to be amodal. In addition, the account of abstract terms seems extremely limited.

Recent work has shown that many problematic aspects of metaphor use and comprehension can be resolved through an account that includes both relevance and perceptual simulation. It has also been shown that metaphors often imply stories, and that stories are often metaphorical. Previous research on narratives has focused primarily on stories that appear either in formal literature or in structured interviews; this essay focuses on stories that occur as an integral part of conversation. It extends recent work on (...) metaphor comprehension to show how use and comprehension of stories in natural, informal conversations can be usefully analyzed in terms of perceptual simulations. Conversely, it extends previous work on storytelling to reveal the metaphorical element in many stories, and to emphasize the social (bonding through shared enjoyment) as well as informative and persuasive functions of storytelling. (shrink)

In this paper I argue that there is a viable simulationist account of emotion attribution. However, I also try to say something specific about the form that this account ought to take. I argue that someone who wants to give by a simulationist account of emotion attribution should focus on similarities between emotions and perceptual judgments.

A fundamental characteristic of human visual perception is the ability to group together disparate elements in a scene and treat them as a single unit. The mechanisms by which humans create such groupings remain unknown, but grouping seems to play an important role in a wide variety of visual phenomena. I propose a neural model of grouping; through top-down control of its circuits, the model implements a grouping strategy that involves both a connection strategy (which elements to connect) and a (...) selection strategy (spatiotemporal properties of a selection signal that segments target elements to facilitate identification). With computer simulations I explain how the circuits work and show how they can account for a wide variety of Gestalt principles of perceptual grouping. Additionally, I extend the model so that it can simulate visual search tasks. I show that when the model uses particular grouping strategies, simulated results closely match empirical results from replication experiments of three visual search tasks. In these experiments, perceptual grouping was induced by proximity and shape similarity (Palmer & Beck, 2007), by the spacing of irrelevant distractors and size similarity (Vickery, 2008), or by the proximity of dots and the proximity and shape similarity of line figures (Trick & Enns, 1997). Thus, I show that the model accounts for a variety of grouping effects and indicates which grouping strategies were likely used to promote performance in three visual search tasks. (shrink)

A theory is proposed that views emotional feelings as pivotal for action control. Feelings of emotions are valued interoceptive signals from the body that become multimodally integrated with perceptual contents from registered and mentally simulated events. During the simulation of a perceptual change from one event to the next, a conative feeling signal is created that codes for the wanting of a specific perceptual change. A wanted perceptual change is weighted more strongly than alternatives, increasing (...) its activation level on the cognitive level and that of associated motor structures that produced this perceptual change in the past. As a consequence, a tendency for action is generated that is directed at the production of the wanted perception. (shrink)

The laws and principles which predict how perceptual qualities can be extracted from the most elementary visual signals were discovered by the Gestalt psychologists(e.g., Wertheimer,1923; Metzger,1930, translated and re-editedbySpillmann in 2009 and2012, respectively). Their seminal work has inspired visual science ever since, andhas led to exciting discoveries which have confirmed the Gestalt idea that the human brain would have an astonishing capacity for selecting and combining critical visual signals to generate output representations for decision making and action. This capacity (...) of selection and integration enables the perception of form and space, and the correct estimation of relative positions, trajectories, and distances of objects represented in planar images. This paper addresses problems of perceptual organization with critical implications for visual interfaces, and the design of surgical simulator platforms in particular. (shrink)

There is considerable evidence that language comprehenders derive lexical‐semantic meaning by mentally simulating perceptual and motor attributes of described events. However, the nature of these simulations—including the level of detail that is incorporated and contexts under which simulations occur—is not well understood. Here, we examine the effects of first‐ versus third‐person perspective on mental simulations during sentence comprehension. First‐person sentences describing physical transfer towards or away from the body (e.g., “You threw the microphone,” “You caught the microphone”) modulated response (...) latencies when responses were made along a front‐back axis, consistent with the action‐sentence compatibility effect (ACE). This effect was not observed for third‐person sentences (“He threw the microphone,” “He caught the microphone”). The ACE was observed when making responses along a left‐right axis for third‐person, but not first‐person sentences. Abstract sentences (e.g., “He heard the message”) did not show an ACE along either axis. These results show that perspective is a detail that is simulated during action sentence comprehension, and that motoric activations are flexible and affected by the pronominal perspective used in the sentence. (shrink)

Computational imagination (CI) conceives imagination as an agent’s simulated sensorimotor interaction with the environment in the absence of sensory feedback, predicting consequences based on this interaction (Marques and Holland in Neurocomputing 72:743–759, 2009). Its bedrock is the simulation hypothesis whereby imagination resembles seeing or doing something in reality as both involve similar neural structures in the brain (Hesslow in Trends Cogn Sci 6(6):242–247, 2002). This paper raises two-forked doubts: (1) neural-level equivalence is escalated to make phenomenological equivalence. Even at (...) an abstract level, many imagined and real actions turn out to be dissimilar. More so, some imagined actions have no corresponding real actions and vice versa, even though neural regions involved in imaginings and real action-perception are the same (Sect. 1). (2) At the implementation level, the hypothesis presents a mutually exclusive view of imagination and perception whereby imagination functions in the absence of the sensory feedback and is action based. Both these issues are contested here: Neither imagination functions in the absence of perception nor all forms of imaginings are action based; it is, rather, about conceiving possibilities which emerge during the perceptual stage itself (Sect. 2). For the modal aspect to arise, it is submitted that an integrative framework is required which Kant can provide for whom imagination is an indispensable part of perception. Kant’s views on concept-formation are presented here to illustrate this aspect (Sect. 3). The Paper is concluded with emphasizing the relevance of Kant’s views to the problems identified in the two sections. (shrink)

There is considerable evidence that language comprehenders derive lexical-semantic meaning by mentally simulating perceptual and motor attributes of described events. However, the nature of these simulations—including the level of detail that is incorporated and contexts under which simulations occur—is not well understood. Here, we examine the effects of first- versus third-person perspective on mental simulations during sentence comprehension. First-person sentences describing physical transfer towards or away from the body modulated response latencies when responses were made along a front-back axis, (...) consistent with the action-sentence compatibility effect. This effect was not observed for third-person sentences. The ACE was observed when making responses along a left-right axis for third-person, but not first-person sentences.sentences did not show an ACE along either axis. These results show that perspective is a detail that is simulated during action sentence comprehension, and that motoric activations are flexible and affected by the pronominal perspective used in the sentence. (shrink)

Although many simulations draw upon only one level of abstraction, the process for generating rich simulations requires a dynamic interplay between abstract and concrete knowledge. A complete model of simulation must account for a mind and brain that can bridge the perceptual with the conceptual, the episodic with the semantic, and the concrete with the abstract.

The paper presents a new proposal how to explain pictorial representation. Starting point is the phenomenological idea that pictures, in the first place, make something visible (instead of being a sign of something). Making something visible is taken as an achievement of the faculty of imagination, and the faculty of imagination in turn is taken to be a variety of our faculty to simulate (referring here to the concept of simulation as it is used in contemporary cognitive psychology, according (...) to which the ability to simulate e.g. other minds is the ability to reenact somebody else’s cognitive processes). Taken together, these pieces come down to the claim that pictorial representation of some x is simulating the perception of x with pictorial means; pictures prompt and guide such a simulation. Crucial is the addition »with pictorial means«. Picture-based simulation has constraints, which differ from those of perception and imagination. Therefore, in picture-based simulation perceptual processes are not simply copied but re-modeled and open to modification. The primary means of such modification are provided by the properties of pictures as material objects, such as their having a frame which encloses a two-dimensional surface. Therefore, being a material object promotes and shapes pictorial representation, instead of hampering it. (shrink)

In this essay, I examine some aspects of the debate between a perceptual model of communication, according to which testimony is a source of knowledge about the communicated fact, and an inferential model of communication, according to which testimony requires from the hearer an inference from the used signs, the speaker’s mental states, and other features of the context. From a reflection on the nature of the capacity for metarepresentation, and its dependence on the capacities of social perception and (...) simulation, I show that the ascription of even complex mental states to the speaker does not preclude communication from functioning on a mode in which the hearer directly inherits the information transmitted by the speaker. Understanding is grounded on the natural tendency to accept what is intelligibly presented to us as true. (shrink)

We present a neural dynamic model that perceptually grounds nested noun phrases, that is, noun phrases that contain further (possibly also nested) noun phrases as parts. The model receives input from the visual array and a representation of a noun phrase from language processing. It organizes a search for the denoted object in the visual scene. The model is a neural dynamic architecture of interacting neural populations which has clear interfaces with perceptual processes. It solves a set of theoretical (...) challenges, including the problem of keeping a nested structure in short-term memory in a way that solves the problem of 2 and massive binding problem emphasized by Jackendoff. The model organizes a search for the objects that are referenced in that structure. We motivate the model, demonstrate simulation results, and discuss how it differs from related models. (shrink)

We present a neural dynamic model that perceptually grounds nested noun phrases, that is, noun phrases that contain further (possibly also nested) noun phrases as parts. The model receives input from the visual array and a representation of a noun phrase from language processing. It organizes a search for the denoted object in the visual scene. The model is a neural dynamic architecture of interacting neural populations which has clear interfaces with perceptual processes. It solves a set of theoretical (...) challenges, including the problem of keeping a nested structure in short-term memory in a way that solves the problem of 2 and massive binding problem emphasized by Jackendoff. The model organizes a search for the objects that are referenced in that structure. We motivate the model, demonstrate simulation results, and discuss how it differs from related models. (shrink)

We introduce a new version of the perceptual retouch model. This model was used for explaining properties of temporal interaction of successive objects in reaching conscious representation. The new model incorporates two interactive binding operations – binding features for objects and binding the bound feature-objects with a large scale oscillatory system that corresponds to perceptual consciousness. Here, the typical result of masking experiments – second object advantage in conscious perception – is achieved by applying the effects of a (...) common synchronizing oscillator with a delay. This delayed modulation of each of the feature-binding first-order oscillators that represent emerging and decaying neural activities of each of the objects guarantees that the oscillating synchrony of the feature-neurons of the following object is higher than the synchrony of the feature-neurons of the first presented object. Thus we model the fact that the following object dominates the preceding object in conscious perception. We also show the capacity of the model to simulate illusory misbinding of features from different objects. The third qualitative effect, the relative release of the first object from backward masking is achieved by priming the non-specific oscillatory modulation ahead in time. (shrink)

The SIMS model claims that it is by means of an embodied simulation that we determine the meaning of an observed smile. This suggests that crucial interpretative work is done in the mapping that takes us from a perceived smile to the activation of one's own facial musculature. How is this mapping achieved? Might it depend upon a prior interpretation arrived at on the basis of perceptual and contextual information?

Various defenses of amodal symbol systems are addressed, including amodal symbols in sensory-motor areas, the causal theory of concepts, supramodal concepts, latent semantic analysis, and abstracted amodal symbols. Various aspects of perceptual symbol systems are clarified and developed, including perception, features, simulators, category structure, frames, analogy, introspection, situated action, and development. Particular attention is given to abstract concepts, language, and computational mechanisms.

Barsalou proposes (sect. 4.1.6) that perceptual symbols play a role in language processing. Data from our laboratory document this role and suggest the sorts of constraints used by simulators during language comprehension.

Mental time travel is defined as projecting the self into the past and the future. Despite growing evidence of the similarities of remembering past and imagining future events, dominant theories conceive of these as distinct capacities. I propose that memory and imagination are fundamentally the same process – constructive episodic simulation – and demonstrate that the ‘simulation system’ meets the three criteria of a neurocognitive system. Irrespective of whether one is remembering or imagining, the simulation system: acts (...) on the same information, drawing on elements of experience ranging from fine-grained perceptual details to coarser-grained conceptual information and schemas about the world; is governed by the same rules of operation, including associative processes that facilitate construction of a schematic scaffold, the event representation itself, and the dynamic interplay between the two ; and is subserved by the same brain system. I also propose that by forming associations between schemas, the simulation system constructs multi-dimensional cognitive spaces, within which any given simulation is mapped by the hippocampus. Finally, I suggest that simulation is a general capacity that underpins other domains of cognition, such as the perception of ongoing experience. This proposal has some important implications for the construct of ‘MTT’, suggesting that ‘time’ and ‘travel’ may not be defining, or even essential, features. Rather, it is the ‘mental’ rendering of experience that is the most fundamental function of this domain-general simulation system enabling humans to re-experience the past, pre-experience the future, and also comprehend the complexities of the present. (shrink)

A metaphor can often create novel features in an object or a situation. This phenomenon has been particularly hard to account for using amodal symbol systems: although highlighting and downplaying can explain the shift of focus, it cannot explain how entirely new features can come about. We suggest here that the dynamism of perceptual symbol systems, particularly the notion of simulator, provides an elegant account of the creativity of metaphor. The elegance lies in the idea that the creation of (...) new features by a metaphor proceeds in the same way as the creation of regular features in a perceptual symbol. (shrink)

Barsalou's theory of perceptual symbol systems is considered from a metacognitive perspective. Two examples are discussed in terms of the proposed perceptual symbol theory. First, recent results in research on feeling-of-knowing judgement are used to argue for a representation of familiarity with input cues. This representation should support implicit memory. Second, the ability of maintaining a theory of other people's beliefs (theory of mind) is considered and it is suggested that a purely simulation-based view is insufficient to (...) explain the available evidence. Both examples characterize areas where Barsalou's theory would benefit from additional detail. (shrink)

The problem of other minds has a distinguished philosophical history stretching back more than two hundred years. Taken at face value, it is an epistemological question: it concerns how we can have knowledge of, or at least justified belief in, the existence of minds other than our own. In recent decades, philosophers, psychologists, neuroscientists, anthropologists and primatologists have debated a related question: how we actually go about attributing mental states to others (regardless of whether we ever achieve knowledge or rational (...) justification in this domain). Until the mid-nineties, the latter debate – which sometimes goes under the name of the “mindreading” debate – was characterized by a fairly clear-cut opposition between two theoretical outlooks: “theory-theory” (TT) and “simulation theory” (ST). Theory-theorists typically argued that we attribute mental states to others on the basis of a “theory of mind” that is either constructed in early infancy and subsequently revised and modified (Gopnik 1996), or else is the result of maturation of innate mindreading “modules” (Baron-Cohen 1995). (shrink)

Theories of embodied cognition hold that the conceptual system uses perceptual simulations for the purposes of representation. A strong prediction is that perceptual phenomena should emerge in conceptual processing, and, in support, previous research has shown that switching modalities from one trial to the next incurs a processing cost during conceptual tasks. However, to date, such research has been limited by its reliance on the retrieval of familiar concepts. We therefore examined concept creation by asking participants to interpret (...) modality-specific compound phrases (i.e., conceptual combinations). Results show that modality switching costs emerge during the creation of new conceptual entities: People are slower to simulate a novel concept (e.g., auditory jingling onion) when their attention has already been engaged by a different modality in simulating a familiar concept (e.g., visual shiny penny). Furthermore, these costs cannot be accounted for by linguistic factors alone. Rather, our findings support the embodied view that concept creation, as well as retrieval, requires situated perceptual simulation. (shrink)

Whether computational algorithms such as latent semantic analysis (LSA) can both extract meaning from language and advance theories of human cognition has become a topic of debate in cognitive science, whereby accounts of symbolic cognition and embodied cognition are often contrasted. Albeit for different reasons, in both accounts the importance of statistical regularities in linguistic surface structure tends to be underestimated. The current article gives an overview of the symbolic and embodied cognition accounts and shows how meaning induction attributed to (...) a specific statistical process or to activation of embodied representations should be attributed to language itself. Specifically, the performance of LSA can be attributed to the linguistic surface structure, more than special characteristics of the algorithm, and embodiment findings attributed to perceptual simulations can be explained by distributional linguistic information. (shrink)

How does the human brain link relational concepts to perceptual experience? For example, a speaker may say “the cup to the left of the computer” to direct the listener's attention to one of two cups on a desk. We provide a neural dynamic account for both perceptual grounding, in which relational concepts enable the attentional selection of objects in the visual array, and for the generation of descriptions of the visual array using relational concepts. In the model, activation (...) in neural populations evolves dynamically under the influence of both inputs and strong interaction as formalized in dynamic field theory. Relational concepts are modeled as patterns of connectivity to perceptual representations. These generalize across the visual array through active coordinate transforms that center the representation of target objects in potential reference objects. How the model perceptually grounds or generates relational descriptions is probed in 104 simulations that systematically vary the spatial and movement relations employed, the number of feature dimensions used, and the number of matching and nonmatching objects. We explain how sequences of decisions emerge from the time‐ and state‐continuous neural dynamics, how relational hypotheses are generated and either accepted or rejected, followed by the selection of new objects or the generation of new relational hypotheses. Its neural realism distinguishes the model from information processing accounts, its capacity to autonomously generate sequences of processing steps distinguishes it from deep neural network accounts. The model points toward a neural dynamic theory of higher cognition. (shrink)

The fundamental relations that underlie cognitive comparisons—“same” and “different”—can be defined at multiple levels of abstraction, which vary in relational complexity. We compared response times to decide whether or not two sequentially‐presented patterns, each composed of two pairs of colored squares, were the same at three levels of abstraction: perceptual, relational, and system (higher order relations). For both 150 ms and 5 s inter‐stimulus intervals (ISIs), both with and without a masking stimulus, decision time increased with level of abstraction. (...) Sameness at lower complexity levels contributed to decisions based on the higher levels. The pattern of comparison times across levelswas not predictable solely from encoding times. The results indicated that relations at multiple levels of complexity can be abstracted and compared in working memory, with higher complexity levels requiring more processing time. We simulated the impact of relational complexity on response time using Learning and Inference with Schemas and Analogies (LISA), a computational model of relational comparisons based on dynamic binding of elements into roles in a relational working memory. (shrink)

The fundamental relations that underlie cognitive comparisons—“same” and “different”—can be defined at multiple levels of abstraction, which vary in relational complexity. We compared response times to decide whether or not two sequentially‐presented patterns, each composed of two pairs of colored squares, were the same at three levels of abstraction: perceptual, relational, and system (higher order relations). For both 150 ms and 5 s inter‐stimulus intervals (ISIs), both with and without a masking stimulus, decision time increased with level of abstraction. (...) Sameness at lower complexity levels contributed to decisions based on the higher levels. The pattern of comparison times across levelswas not predictable solely from encoding times. The results indicated that relations at multiple levels of complexity can be abstracted and compared in working memory, with higher complexity levels requiring more processing time. We simulated the impact of relational complexity on response time using Learning and Inference with Schemas and Analogies (LISA), a computational model of relational comparisons based on dynamic binding of elements into roles in a relational working memory. (shrink)

The present paper introduces a simple framework for modeling the relationship between environmental states, perceptual states, and action. The framework represents situations where an agent’s perceptual state forms the basis for choosing an action, and what action the agent performs determines the agent’s payoff, as a function of the environmental conditions in which the action is performed. The framework is used as the basis for a simulation study of the sorts of correspondence between perceptual and environmental (...) states that are important for successful navigation of the world. Some of the results are surprising and conflict with long held views about the kind of perception-to-environment correspondence that is important for knowledge of the world. The results also raise doubts concerning the view that our perceptual states provide a basis for knowledge of the real structure of the external world. (shrink)

The main aim of this article is to present and defend a thesis according to which conceptual representations of some types of mental states are encoded in the same neural structures that underlie the first-personal experience of those states. To support this proposal here, I will put forth a novel account of the cognitive function played by ‘shared representations’ of emotions and bodily sensations, i.e. neural structures that are active when one experiences a mental state of a certain type as (...) well as when one observes someone else experiencing a state of the same type. I will argue that shared representations in fact constitute vehicles of certain mental state concepts (more precisely, concepts of specific types of emotions and somatosensory states). The main line of arguing for this will consist in showing that shared representations exhibit specific, ‘conceptual’ functional properties:(1) causal effect on forming metacognitive judgments, (2) cognitive penetrability, (3) diversity of input types. (shrink)

Stich & Ravenscroft (1994) have argued that (contrary to most people's initial assumptions) a simulation account of folk psychology may be consistent with eliminative materialism, but they fail to bring out the full complexity or the potential significance of the relationship. Contemporary eliminativism (particularly in the Churchland version) makes two major claims: the first is a rejection of the orthodox assumption that realistically construed propositional attitudes are fundamental to human cognition; the second is the suggestion that with the advancement (...) of scientific understanding of the mind it will be possible to entirely eliminate the mentalistic and intentional from our ontology, thus dissolving the mind-body problem. The first claim (which has been argued in detail) supplies the principal grounds for accepting the second, much more ambitious and significant, claim. Robert Gordon's (1995, 1996, 2000) radical simulation theory of "folk psychology", proposed initially (Gordon, 1986) as an alternative to "theory theory" accounts of self and interpersonal understanding, but subsequently developing into a quite general challenge to symbolic computational accounts of mind, is not merely consistent with, but actually provides considerable additional support for, the first eliminativist claim. However, although radical simulationism has no use for reified propositional attitudes, it relies on another family of mentalistic and intentional notions, including perspective taking, "seeing as", pretending, imagery, and, most centrally, imagination. It is thus inconsistent with eliminativist metaphysical ambitions. Nevertheless, from this perspective the mind-body problem is transformed. Its solution no longer depends on accounting directly for the intentionality of the attitudes, but rather on accounting for the intentionality of imagination. Although standard accounts of imagination derive its intentionality from that of the attitudes, the recently proposed "perceptual activity" theory of imagery and imagination (Thomas, 1999) can provide a direct account of the intentionality of imagination that is consistent with physicalism.. (shrink)