Since listeners usually look at the speaker's face, gestural information has to be absorbed through peripheral visual perception. In the literature, it has been suggested that listeners look at gestures under certain circumstances: 1) when the articulation of the gesture is peripheral; 2) when the speech channel is insufficient for comprehension; and 3) when the speaker him- or herself indicates that the gesture is worthy of attention. The research here reported employs eye tracking techniques to study the perception of gestures (...) in face-to-face interaction. The improved control over the listener's visual channel allows us to test the validity of the above claims. We present preliminary findings substantiating claims 1 and 3, and relate them to theoretical proposals in the literature and to the issue of how visual and cognitive attention are related. (shrink)

"This is the most systematic discussion of semiotics yet published." --Choice -/- "A bravura performance." --Thomas Sebeok -/- "Nöth's handbook is an outstanding encyclopedia that provides first-rate information on many facets of sign-related studies, research results, and applications." --Social Sciences in General.

Reasoning about bedrock abstract concepts such as time, number, and valence relies on spatial metaphor and often on multiple spatial metaphors for a single concept. Previous research has documented, for instance, both future-in-front and future-to-right metaphors for time in English speakers. It is often assumed that these metaphors, which appear to have distinct experiential bases, remain distinct in online temporal reasoning. In two studies we demonstrate that, contra this assumption, people systematically combine these metaphors. Evidence for this combination was found (...) in both directly elicited and spontaneous co-speech gestures about time. These results provide first support for the hypothesis that the metaphorical representation of time, and perhaps other abstract domains as well, involves the continuous co-activation of multiple metaphors rather than the selection of only one. (shrink)

Flexibility in conceptual projection constitutes one of the most challenging issues in the embodiment and conceptual metaphor literatures. We sketch a theoretical proposal that places the burden of the explanation on attentional dynamics in interaction with mental models in working memory that are constrained to be maximally coherent. A test of this theory is provided in the context of the conceptual projection of time onto the domain of space. Participants categorized words presented at different spatial locations (back–front, left–right) as referring (...) to the past or to the future. Responses were faster when the irrelevant word location was congruent with the back–past, front–future metaphoric mapping. Moreover, when a new highly task-relevant spatial frame of reference was introduced, it changed the projection of past and future onto space in a way that was congruent with the new frame (past was now projected to left space and future to right space), as predicted by the theory. This study shows that there is substantial flexibility in conceptual projection and opens a venue to study metaphoric variation across tasks, individuals, and cultures as the result of attentional dynamics. (shrink)

Most research on metaphors that construe time as motion (motion metaphors of time) has focused on the question of whether it is the times or the person experiencing them (ego) that moves. This paper focuses on the equally important distinction between metaphors that locate times relative to ego (the ego-based metaphors Moving Ego and Moving Time) and a metaphor that locates times relative to other times (sequence is relative position on a path). Rather than a single abstract target domain TIME, (...) these two kinds of temporal metaphor metaphorize different kinds of temporal concept—perspective-specific vs. perspective-neutral temporal concepts. Recognition of this distinction enhances the explanatory potential of conceptual metaphor theory (Lakoff and Johnson 1980). An example involves the interaction of deixis and the temporal reference crosslinguistically of vocabulary with the spatial meanings in-front and behind. More generally, this approach refines our ability to describe the temporal concepts involved in motion metaphors of time. Such temporal concepts are present not only in the target domains, but also in the source domains of motion metaphors of time, where we find space-to-time metonymy, which may play a role in motivating the metaphors. In order to distinguish such metonymy from metaphor, we need to characterize metaphor as a mapping across frames rather than domains. (shrink)

Journal Name: Semiotica Issue: Ahead of print.

How do we understand time and other entities we can neither touch nor see? One possibility is that we tap into our concrete, experiential knowledge, including our understanding of physical space and motion, to make sense of abstract domains such as time. To examine how pervasive an aspect of cognition this is, we investigated whether thought about a nonliteral type of motion called fictive motion (FM; as in The road runs along the coast) can influence thought about time. Our results (...) suggest that FM uses the same structures evoked in understanding literal motion, and that these literal aspects of FM influence temporal reasoning. (shrink)

The now-classic Metaphors We Live By changed our understanding of metaphor and its role in language and the mind. Metaphor, the authors explain, is a fundamental mechanism of mind, one that allows us to use what we know about our physical and social experience to provide understanding of countless other subjects. Because such metaphors structure our most basic understandings of our experience, they are "metaphors we live by"--metaphors that can shape our perceptions and actions without our ever noticing them. In (...) this updated edition of Lakoff and Johnson's influential book, the authors supply an afterword surveying how their theory of metaphor has developed within the cognitive sciences to become central to the contemporary understanding of how we think and how we express our thoughts in language. (shrink)

This study investigates whether addressees visually attend to speakers’ gestures in interaction and whether attention is modulated by changes in social setting and display size. We compare a live face-to-face setting to two video conditions. In all conditions, the face dominates as a fixation target and only a minority of gestures draw fixations. The social and size parameters affect gaze mainly when combined and in the opposite direction from the predicted with fewer gestures fixated on video than live. Gestural holds (...) and speakers’ gaze at their own gestures reliably attract addressees’ fixations in all conditions. The attraction force of holds is unaffected by changes in social and size parameters, suggesting a bottom-up response, whereas speaker-fixated gestures draw significantly less attention in both video conditions, suggesting a social effect for overt gaze-following and visual joint attention. The study provides and validates a video-based paradigm enabling further experimental but ecologically valid explorations of cross-modal information processing. (shrink)

In his 1903 Syllabus, Charles S. Peirce makes a distinction between icons and iconic signs, or hypoicons, and briefly introduces a division of the latter into images, diagrams, and metaphors. Peirce scholars have tried to make better sense of those concepts by understanding iconic signs in the context of the ten classes of signs described in the same Syllabus. We will argue, however, that the three kinds of hypoicons can better be understood in the context of Peirce's sixty-six classes of (...) signs. We analyze examples of hypoicons taken from the field of information design, describing them in the framework of the sixty-six classes, and discuss the consequences of those descriptions to the debate about the order of determination of the 10 trichotomies that form those classes. (shrink)

Across cultures, people employ space to construct representations of time. English exhibits two deictic space–time metaphors: the “moving ego” metaphor conceptualizes the ego as moving forward through time and the “moving time” metaphor conceptualizes time as moving forward towards the ego. Earlier research investigating the psychological reality of these metaphors has shown that engaging in certain types of spatial-motion thinking may influence how people reason about events in time. More recently, research has shown that people’s interactions with cultural artifacts may (...) also influence their representations of time. Extending research on space–time mappings in new directions, three experiments investigated the role of cultural artifacts, namely calendars and clocks, in the interpretation of metaphorical expressions about time. Taken together, the results provide initial evidence that, in their interpretation of ambiguous metaphorical expressions about time, people automatically access and use spatial representations of absolute time, whereby moving forward in space corresponds with moving later in time. Moreover, asking participants to use a reverse space–time mapping causes interference, which is reflected through their temporal reasoning. (shrink)

Drawing on recent work in developmental and comparative psychology, this paper argues that demonstratives function to coordinate the interlocutors' joint focus of attention, which is one of the most basic functions of human communication. The communicative importance of demonstratives is reflected in a number of properties that together characterize them as a particular word class: In contrast to other closed-class expressions, demonstratives are universal, they are generally so old that their roots cannot be traced back to other linguistic items, they (...) are among the earliest words that children learn, and they are closely tied to a particular gesture. Moreover, demonstratives play an important role in the organization of discourse and the diachronic evolution of grammar, which arguably is also motivated by their communicative function to establish joint attention. (shrink)

Do English speakers think about time the way they talk about it? In spoken English, time appears to flow along the sagittal axis (front/back): the future is ahead and the past is behind us. Here we show that when asked to gesture about past and future events deliberately, English speakers often use the sagittal axis, as language suggests they should. By contrast, when producing co-speech gestures spontaneously, they use the lateral axis (left/right) overwhelmingly more often, gesturing leftward for earlier times (...) and rightward for later times. This left-right mapping of time is consistent with the flow of time on calendars and graphs in English-speaking cultures, but is completely absent from conventional spoken metaphors. English speakers gesture on the lateral axis even when they are using front/back metaphors in their co-occurring speech. This speech-gesture dissociation is not due to any lack of lexical or constructional resources to spatialize time laterally in language, nor to any lack of physical resources to spatialize time sagittally in gesture. We propose that when speakers are describing sequences of events, they often use neither the Moving Ego nor Moving Time perspectives. Rather, they adopt a “Moving Attention” perspective, which is grounded in patterns of interaction with cultural artifacts, not in patterns of interaction with the natural environment. We suggest possible pragmatic, kinematic, and mnemonic motivations for the use of a lateral mental timeline in gesture and in thought. Gestures reveal an implicit spatial conceptualization of time that cannot be inferred from language. (shrink)

This paper presents a proposal for the description of gesture phases derived from articulatory characteristics observable in their execution. Based on the results of an explorative study examining the execution of gesture phases of ten German speakers, the paper presents two sets of articulatory features, i.e., distinctive and additional features by which gesture phases are characterized from a context-independent and context-sensitive point of view. It will be shown that gesture phases show a particular distribution of the features, thus distinguishing one (...) phase from another. Furthermore, changes in the execution of phases in linear successions can be described by means of features. Contrary to other accounts, whose focus on gesture phases is primarily in relation to speech and/or adjacent phases, this proposal concentrates on the visible physical characteristics of gesture phases. (shrink)

Past research has suggested that those spontaneous movements of the human hand made during talk convey significant semantic information over and above the speech, at least when the unit of speech analyzed is the individual clause. However, no previous research has tested whether this information is represented linguistically elsewhere in the narrative . The first study, reported here, uses an experimental procedure to identify which specific imagistic gestures add semantic information to the speech. The second study analyzes whether the specific (...) gestures still do this when respondents hear the whole narrative. It was found that two thirds of the semantic information, thought to be carried by the gestures, is, in fact, represented in the linguistic discourse, or is inferable from it. However, one third of the additional semantic information contained in the gestures is not represented linguistically in the narrative nor is it inferable from it. In other words, a proportion of the imagistic gestures that accompany speech are absolutely critical to semantic communication. (shrink)

History and Classics of Modern Semiotics -- Sign and Meaning -- Semiotics, Code, and the Semiotic Field -- Language and Language-Based Codes -- From Structuralism to Text Semiotics: Schools and Major Figures -- Text Semiotics: The Field -- Nonverbal Communication -- Aesthetics and Visual Communication.

In Gesture and Thought he brings together years of this research, arguing that gesturing, an act which has been popularly understood as an accessory to speech, ...