This paper examines the meaningfulness of pointing in great apes. We appeal to Hannah Ginsborg’s conception of primitive normativity, which provides an adequate criterion for establishing whether a response is meaningful, and we attempt to make room for a conception according to which there is no fundamental difference between the responses of human infants and those of other great apes to pointing gestures. This conception is an alternative to Tomasello’s view that pointing gestures and reactions to them (...) reveal a fundamental difference between humans and other apes. (shrink)

In contrast to apes’ seemingly sophisticated skill at producing pointing gestures referentially, the comprehension of other individual’s pointing gestures as a source of indexical information seems to be less pronounced.One reason for apes’ difficulty at comprehending pointing gestures might be that in former studies they were mainly confronted with human declarative pointing gestures, whereas apes have largely been shown to point imperatively and towards humans. In the present study bonobos, chimpanzees and orangutans were confronted with a (...) conspecific’s imperative pointing gesture in a competitive context, therefore mirroring former studies that have investigated apes’ skills at producing these gestures.However, apes in the present study did not use their conspecific’s pointing gestures. Apes have been shown to use indexical information when provided noncommunicatively and to interpret other individuals’ actions in terms of motives. Thus, it is discussed whether apes treat a pointing gesture as intentional act of indexical reference. (shrink)

In contrast to apes' seemingly sophisticated skill at producing pointing gestures referentially, the comprehension of other individual's pointing gestures as a source of indexical information seems to be less pronounced.One reason for apes' difficulty at comprehending pointing gestures might be that in former studies they were mainly confronted with human declarative pointing gestures, whereas apes have largely been shown to point imperatively and towards humans. In the present study bonobos, chimpanzees and orangutans were confronted with a (...) conspecific's imperative pointing gesture in a competitive context, therefore mirroring former studies that have investigated apes' skills at producing these gestures.However, apes in the present study did not use their conspecific's pointing gestures. Apes have been shown to use indexical information when provided noncommunicatively and to interpret other individuals' actions in terms of motives. Thus, it is discussed whether apes treat a pointing gesture as intentional act of indexical reference. (shrink)

Two‐ and 3‐year‐old children (N = 96) were tested in an object‐choice task with video presentations of peer and adult partners. An immersive, semi‐interactive procedure enabled both the close matching of adult and peer conditions and the combination of participants’ choice behavior with looking time measures. Children were more likely to use information provided by adults. As the effect was more pronounced in the younger age‐group, the observed bias may fade during toddlerhood. As there were no differences in children’s propensity (...) to follow peer and adult gestures with their gaze, these findings provide some of the earliest evidence to date that young children take an interlocutor’s age into account when judging ostensively communicated testimony. (shrink)

Human languages typically employ a variety of spatial metaphors for time (e.g., “I'm looking forward to the weekend”). The metaphorical grounding of time in space is also evident in gesture. The gestures that are performed when talking about time bolster the view that people sometimes think about regions of time as if they were locations in space. However, almost nothing is known about the development of metaphorical gestures for time, despite keen interest in the origins of space–time metaphors. In (...) this study, we examined the gestures that English‐speaking 6‐to‐7‐year‐olds, 9‐to‐11‐year‐olds, 13‐to‐15‐year‐olds, and adults produced when talking about time. Participants were asked to explain the difference between pairs of temporal adverbs (e.g., “tomorrow” versus “yesterday”) and to use their hands while doing so. There was a gradual increase across age groups in the propensity to produce spatial metaphorical gestures when talking about time. However, even a substantial majority of 6‐to‐7‐year‐old children produced a spatial gesture on at least one occasion. Overall, participants produced fewer gestures in the sagittal (front‐back) axis than in the lateral (left‐right) axis, and this was particularly true for the youngest children and adolescents. Gestures that were incongruent with the prevailing norms of space–time mappings among English speakers (leftward and backward for past; rightward and forward for future) gradually decreased with increasing age. This was true for both the lateral and sagittal axis. This study highlights the importance of metaphoricity in children's understanding of time. It also suggests that, by 6 to 7 years of age, culturally determined representations of time have a strong influence on children's spatial metaphorical gestures. (shrink)

Journal Name: Cognitive Linguistics Issue: Ahead of print.

Temporal perspectives allow us to place ourselves and temporal events on a timeline, making it easier to conceptualize time. This study investigates how we take different temporal perspectives in our temporal gestures. We asked participants (n = 36) to retell temporal scenarios written in the Moving‐Ego, Moving‐Time, and Time‐Reference‐Point perspectives in spontaneous and encouraged gesture conditions. Participants took temporal perspectives mostly in similar ways regardless of the gesture condition. Perspective comparisons showed that temporal gestures of our participants resonated (...) better with the Ego‐ (i.e., Moving‐Ego and Moving‐Time) versus Time‐Reference‐Point distinction instead of the classical Moving‐Ego versus Moving‐Time contrast. Specifically, participants mostly produced more Moving‐Ego and Time‐Reference‐Point gestures for the corresponding scenarios and speech; however, the Moving‐Time perspective was not adopted more than the others in any condition. Similarly, the Moving‐Time gestures did not favor an axis over the others, whereas Moving‐Ego gestures were mostly sagittal and Time‐Reference‐Point gestures were mostly lateral. These findings suggest that we incorporate temporal perspectives into our temporal gestures to a considerable extent; however, the classical Moving‐Ego and Moving‐Time classification may not hold for temporal gestures. (shrink)

Name der Zeitschrift: Cognitive Linguistics Jahrgang: 27 Heft: 1 Seiten: 67-100.

I have two objectives in this article. The first is methodological: I elaborate a minimal phenomenological method and attempt to show its importance in studies of infant behavior. The second objective is substantive: Applying the minimal phenomenological approach, combined with Meltzoff’s “like-me” developmental framework, I propose the hypothesis that infants learn the pointing gesture at least in part through imitation. I explain how developments in sensorimotor ability (posture, arm and hand control and coordination, and locomotion) in the first (...) year of life prepare the infant for acquiring the pointing gesture. The former may directly enable the latter by allowing the infant to experience its own body as being “like those” of others, thus allowing it to imitatively appropriate a broader range of adult behavior. My proposal emphasizes the embodiment of mind in the development of cognition, contrary to latent dualistic tendencies in some developmental literature. (shrink)

David McNeill, a pioneer in the ongoing study of the relationship between gesture and language, here argues that gestures are active participants in both speaking and thinking. He posits that gestures are key ingredients in an “imagery-language dialectic” that fuels speech and thought. The smallest unit of this dialectic is the growth point, a snapshot of an utterance at its beginning psychological stage. In _Gesture and Thought,_ the central growth point comes from a Tweety Bird cartoon. Over the course (...) of twenty-five years, the McNeill Lab showed this cartoon to numerous subjects who spoke a variety of languages, and a fascinating pattern emerged. The shape and timing of gestures depends not only on what speakers see but on what they take to be distinctive; this, in turn, depends on the context. Those who remembered the same context saw the same distinctions and used similar gestures; those who forgot the context understood something different and changed gestures or used none at all. Thus, the gesture becomes part of the growth point—the building block of language and thought. _Gesture and Thought _is an ambitious project in the ongoing study of how we communicate and how language is connected to thought. (shrink)

Most language learners have difficulties acquiring the phonemes of a second language (L2). Unfortunately, they are often judged on their L2 pronunciation, and segmental inaccuracies contribute to miscommunication. Therefore, we aim to determine how to facilitate phoneme acquisition. Given the close relationship between speech and co-speech gesture, previous work unsurprisingly reports that gestures can benefit language acquisition, e.g., in (L2) word learning. However, gesture studies on L2 phoneme acquisition present contradictory results, implying that both specific properties of gestures (...) and phonemes used in training, and their combination, may be relevant. We investigated the effect of phoneme and gesture complexity on L2 phoneme acquisition. In a production study, Dutch natives received instruction on the pronunciation of two Spanish phonemes, /u/ and /θ/. Both are typically difficult to produce for Dutch natives because their orthographic representation differs between both languages. Moreover, /θ/ is considered more complex than /u/, since the Dutch phoneme inventory contains /u/ but not /θ/. The instruction participants received contained Spanish examples presented either via audio-only, audio-visually without gesture, audio-visually with a simple, pointing gesture, or audio-visually with a more complex, iconic gesture representing the relevant speech articulator(s). Preceding and following training, participants read aloud Spanish sentences containing the target phonemes. In a perception study, Spanish natives rated the target words from the production study on accentedness and comprehensibility. Our results show that combining gesture and speech in L2 phoneme training can lead to significant improvement in L2 phoneme production, but both gesture and phoneme complexity affect successful learning: Significant learning only occurred for the less complex phoneme /u/ after seeing the more complex iconic gesture, whereas for the more complex phoneme /θ/, seeing the more complex gesture actually hindered acquisition. The perception results confirm the production findings and show that items containing /θ/ produced after receiving training with a less complex pointing gesture are considered less foreign-accented and more easily comprehensible as compared to the same items after audio-only training. This shows that gesture can facilitate task performance in L2 phonology acquisition, yet complexity affects whether certain gestures work better for certain phonemes than others. (shrink)

Gestures is a collection of essays that proposes a daring and ambitious new conception of human behavior. Defining gesture as “a movement of the body or of a tool attached to the body for which there is no satisfactory causal explanation,” Flusser moves around the topic from different points of view, angles and distances: sometimes he zooms in on a modest, ordinary movement like taking a photograph, shaving, or smoking a pipe. Sometimes he pulls back to look at something (...) as vast and varied as human “making,” embracing everything from the fashioning of simple tools to mass manufacturing. Holding firmly to basic phenomenological principles – that consciousness is always consciousness of something, that we know others by reference to ourselves, he claims that we constantly “read” states of mind, i.e. thoughts, intentions, emotions, from gestures; still we lack a theory about how this happens. Gestures takes a first step. It offers alternatives to theories by now so veiled by habit and myth that we are hardly conscious of them, and so hardly realize that they are failing. These include the assumption that we can “know” something without being affected by it, the belief that science is value-free, and the common conviction that science and art are fundamentally different activities. (shrink)

The framework of depicting put forward byClark (2016)offers a schematic vantage point from which to examine iconic language use. Confronting the framework with empirical data, we consider some of its key theoretical notions. Crucially, by reconceptualizing the typology of depictions, we identify an overlooked domain in the literature: “speech-embedded nonverbal depictions,” namely cases where meaning is communicated iconically, nonverbally, and without simultaneously co-occurring speech. In addition to contextualizing the phenomenon in relation to existing research, we demonstrate, with examples from American (...) TV talk shows, how such depictions function in real-life language use, offering a brief sketch of their complexities and arguing also for their theoretical significance. (shrink)

Concepts such as “symbolism” and “symbolic cognition” often remain unspecified in discussions the symbolic capacities of earlier hominins. In this paper, I use conceptual tools from phenomenology to reflect on the origins of early symbolic cognition. In particular, I discuss the possible early use of pointing gestures around the time of the earliest known stone tool industries. I argue that unlike more basic social acts such as expression, gaze following, and attention-getters, which are used by extant non-human great apes, (...) communicative pointing involves key elements that are characteristic of symbolic cognition. In particular, it involves “third order intentionality” as well as “shared practice horizons”: shared frameworks of understanding which are required for the interpretation of communicative acts whose meaning is not codified indexically or iconically in the signaling behavior. In the final part, I briefly review some indications for the use of pointing gestures around the time of the Lomekwian and Oldowan industries, as a way to sustain cooperation and possibly learning by instruction. It is suggested that pointing is more complex than is standardly acknowledged, and that it may have been an important communicative act for Early Stone Age hominins in transitioning to more fully symbolic speech capacities. (shrink)

Two eye-tracking experiments investigated how and when pointing gestures and location descriptions affect target identification. The experiments investigated the effect of gestures and referring expressions on the time course of fixations to the target, using videos of human gestures and human voice, and animated gestures and synthesized speech. Ambiguous, yet informative pointing gestures elicited attention and facilitated target identification, akin to verbal location descriptions. Moreover, target identification was superior when both pointing gestures and verbal location descriptions were (...) used. These findings suggest that gesture not only operates as a context to verbal descriptions, or that verbal descriptions operate as a context to gesture, but that they complement one another in reference resolution. (shrink)

The article argues in favour of a pragmatist enactive interpretation of the emergence of the symbolic and contentful mind from a basic form of social communicative interaction in which basic cognitive capacities are involved. Through a critical overview of Radical Enactivists (RECers)’ view about language, the article focuses on Mead’s pragmatist behavioural theory of meaning that refers to the gestural conversation as the origin of the evolution of linguistic conversation. The article develops as follows. After exposing the main elements of (...) REC’s theory of cognition and language that involve the construction of a theory of natural signs (teleosemiotics) and basic directionality (Ur-intentionality), some critical points of Hutto and Myin’s proposal will be highlighted. To foster a continuist perspective of language, the behavioural theory of meaning and language that Mead develops from the notion of gesture will be analysed. His theory is akin to REC and could augment the bare bones of REC’s sketched perspective, helping to include Ur-intentionality in a broader non-dualistic phylogenetic and ontogenetic theory of symbolic language from gestural communication, thus helping to overcome the distinction between a content-less intentionality and a content-involving intentionality, i.e., a semantic propositional intentionality. Furthermore, a recent revival of Mead’s theory testifies to its up-to-date relevance to explain the innate social dimension of human and non-human animals, and the human communicative capacity through the conditioning of bio-social canons and structures. (shrink)

When asked to explain their solutions to a problem, both adults and children gesture as they talk. These gestures at times convey information that is not conveyed in speech and thus reveal thoughts that are distinct from those revealed in speech. In this study, we use the classic Tower of Hanoi puzzle to validate the claim that gesture and speech taken together can reflect the activation of two cognitive strategies within a single response. The Tower of Hanoi is (...) a well‐studied puzzle, known to be most efficiently solved by activating subroutines at theoretically defined choice points. When asked to explain how they solved the Tower of Hanoi puzzle, both adults and children produced significantly more gesture‐speech mismatches—explanations in which speech conveyed one path and gesture another—at these theoretically defined choice points than they produced at non‐choice points. Even when the participants did not solve the problem efficiently, gesture could be used to indicate where the participants were deciding between alternative paths. Gesture can, thus, serve as a useful adjunct to speech when attempting to discover cognitive processes in problem‐solving. (shrink)

This paper presents a proposal for the description of gesture phases derived from articulatory characteristics observable in their execution. Based on the results of an explorative study examining the execution of gesture phases of ten German speakers, the paper presents two sets of articulatory features, i.e., distinctive and additional features by which gesture phases are characterized from a context-independent and context-sensitive point of view. It will be shown that gesture phases show a particular distribution of the (...) features, thus distinguishing one phase from another. Furthermore, changes in the execution of phases in linear successions can be described by means of features. Contrary to other accounts, whose focus on gesture phases is primarily in relation to speech and/or adjacent phases, this proposal concentrates on the visible physical characteristics of gesture phases. (shrink)

Gestural viewpoint research suggests that several dimensions determine which perspective a narrator takes, including properties of the event described. Events can evoke gestures from the point of view of a character , an observer , or both perspectives. CVPT and OVPT gestures have been compared to constructed action and classifiers in signed languages. We ask how CA and CL, as represented in ASL productions, compare to previous results for CVPT and OVPT from English-speaking co-speech gesturers. Ten ASL signers described cartoon (...) stimuli from Parrill . Events shown by Parrill to elicit a particular gestural strategy were coded for signers' instances of CA and CL. CA was divided into three categories: CA-torso, CA-affect, and CA-handling. Signers used CA-handling the most when gesturers used CVPT exclusively. Additionally, signers used CL the most when gesturers used OVPT exclusively and CL the least when gesturers used CVPT exclusively. (shrink)

Gestures can serve as external representations of abstract concepts which may be otherwise difficult to illustrate. Gestures often accompany verbal statement as an embodiment of mental models that augment the communication of ideas, concepts or envisioned shapes of products. A gesture is also an indicator of the subject and context of the issue under discussion. We argue that if gestures can be identified and formalized they can be used as a knowledge indexing and retrieval tool and can prove to (...) be useful access point into unstructured digital video data. We present a methodology and a prototype, called I-Gesture that allows users to (1) define a vocabulary of gestures for a specific domain, (2) build a digital library of the gesture vocabulary, and (3) mark up entire video streams based on the predefined vocabulary for future search and retrieval of digital content from the archive. I-Gesture methodology and prototype are illustrated through scenarios where it can be utilized. The paper concludes with results of evaluation experiments with I-Gesture using a test bed of design-construction projects. (shrink)

Deictic (or pointing) gestures are traditionally known to have a simple function: to supply something as the referent of a demonstrative linguistic expression. I argue that deixis can have a more complex function. A deictic gesture can be used to _say something_ in conversation and can thereby become a full discourse move in its own right. To capture this phenomenon, which I call _rich demonstration_, I present an update semantics on which deictic gestures can indicate situations from a (...) conversation’s context and those situations coherently connect to prior discourse to generate information. (shrink)

The paper deals with the theory of gesture from the point of view of relational sociology. On the one hand, the thesis of the ‘complete gesture’ developed by Giovanni Maddalena is appreciated as a significant step forward from classical pragmatism. On the other hand, since theories based essentially on phenomenology and semiotics are at risk of nominalism and constructivism, if we want to understand the gesture from a critical realistic perspective, we need to complement the theory of (...) gesture with a relational social ontology. This means that the theory of the gesture as action (unit act) must be placed within an ontological and epistemological framework, in which Peirce’s triangle is related to the latent value of the real as indicated by the sign. A relational alternative to Peirce's semiotic triangle is presented here with the aim of connecting the sign of the gesture to the underlying reality. (shrink)

Throughout his career, the influential new media theorist Vilém Flusser kept the idea of gesture in mind: that people express their being in the world through a sweeping range of movements. He reconsiders familiar actions—from speaking and painting to smoking and telephoning—in terms of particular movement, opening a surprising new perspective on the ways we share and preserve meaning. A gesture may or may not be linked to specialized apparatus, though its form crucially affects the person who makes (...) it. These essays, published here as a collection in English for the first time, were written over roughly a half century and reflect both an eclectic array of interests and a durable commitment to phenomenological thought. Defining gesture as “a movement of the body or of a tool attached to the body for which there is no satisfactory causal explanation,” Flusser moves around the topic from diverse points of view, angles, and distances: at times he zooms in on a modest, ordinary movement such as taking a photograph, shaving, or listening to music; at others, he pulls back to look at something as vast and varied as human “making,” embracing everything from the fashioning of simple tools to mass manufacturing. But whatever the gesture, Flusser analyzes it as the expression of a particular form of consciousness, that is, as a particular relationship between the world and the one who gestures. (shrink)

Several cognitive accounts of human communication argue for a language-independent, prelinguistic basis of human communication and language. The current study provides evidence for the universality of a prelinguistic gestural basis for human communication. We used a standardized, semi-natural elicitation procedure in seven very different cultures around the world to test for the existence of preverbal pointing in infants and their caregivers. Results were that by 10–14 months of age, infants and their caregivers pointed in all cultures in the same (...) basic situation with similar frequencies and the same proto-typical morphology of the extended index finger. Infants’ pointing was best predicted by age and caregiver pointing, but not by cultural group. Further analyses revealed a strong relation between the temporal unfolding of caregivers’ and infants’ pointing events, uncovering a structure of early prelinguistic gestural conversation. Findings support the existence of a gestural, language-independent universal of human communication that forms a culturally shared, prelinguistic basis for diversified linguistic communication. (shrink)

The tradeoff hypothesis in the speech–gesture relationship claims that (a) when gesturing gets harder, speakers will rely relatively more on speech, and (b) when speaking gets harder, speakers will rely relatively more on gestures. We tested the second part of this hypothesis in an experimental collaborative referring paradigm where pairs of participants (directors and matchers) identified targets to each other from an array visible to both of them. We manipulated two factors known to affect the difficulty of speaking to (...) assess their effects on the gesture rate per 100 words. The first factor, codability, is the ease with which targets can be described. The second factor, repetition, is whether the targets are old or new (having been already described once or twice). We also manipulated a third factor, mutual visibility, because it is known to affect the rate and type of gesture produced. None of the manipulations systematically affected the gesture rate. Our data are thus mostly inconsistent with the tradeoff hypothesis. However, the gesture rate was sensitive to concurrent features of referring expressions, suggesting that gesture parallels aspects of speech. We argue that the redundancy between speech and gesture is communicatively motivated. (shrink)

More than a century ago, reviewing the raging controversy over pragmatism, Jean Bourdeau wrote that “Pragmatism is an Anglo-Saxon reaction against the intellectualism and rationalism of the Latin mind […] It is a philosophy without words, a philosophy of gestures and of acts, which abandons what is general and holds only to what is particular” (Trans. William James, W:MT, 113). Bourdeau certainly missed the point of the first pragmatist revolution, but it can also be argued that, ironically,...

This article brings a phenomenological perspective to the question of how bodily and inter-bodily experience is involved in interacting via audio-visual media like videoconferencing platforms. Contemporary discussions in interaction studies point to a certain suspension of bodily involvement in these mediated interactions, which leads to a visible loss of function in the case of gestures. Such observations have led phenomenologists to voice concern as to whether phenomenology is indeed still suited to account for the “digital world” in general. The following (...) article addresses these concerns by first making the case for a phenomenological understanding of gestures, which develops Merleau-Ponty’s notion of intercorporeality by drawing from an intersubjective reading of Husserl’s threefold analysis of the body (aesthesiological, kinaesthetic and orientational). Subsequently, these reflections are used to describe the modifications, which occur when interacting “through” the screen in videoconferences, by showing that they are not just privative in nature. (shrink)

Two central assumptions of current models of language acquisition were addressed in this study: (1) knowledge of linguistic structure is "mapped onto" earlier forms of non-linguistic knowledge; and (2) acquiring a language involves a continuous learning sequence from early gestural communication to linguistic expression. The acquisition of the first and second person pronouns ME and YOU was investigated in a longitudinal study of two deaf children of deaf parents learning American Sign Language (ASL) as a first language. Personal pronouns in (...) ASL are formed by pointing directly to the addressee (YOU) or self (I or ME), rather than by arbitrary symbols. Thus, personal pronouns in ASL resemble paralinguistic gestures that commonly accompany speech and are used prelinguistically by both hearing and deaf children beginning around 9 months. This provides a means for investigating the transition from prelinguistic gestural to linguistic expression when both gesture and language reside in the same modality.\nThe results indicate that deaf children acquired knowledge of personal pronouns over a period of time, displaying errors similar to those of hearing children despite the transparency of the pointing gestures. The children initially (ages 10 and 12 months) pointed to persons, objects, and locations. Both children then exhibited a long avoidance period, during which one function of the pointing gesture (pointing to self and others) dropped out completely. During this period their language and cognitive development were otherwise entirely normal, and they continued to use other types of pointing (e.g., to objects). When pointing to self and others returned, it was marked with errors typical of hearing children; one child exhibited consistent pronoun reversal errors, thinking the YOU point referred to herself, while the other child exhibited reversal errors inconsistently. Evidence from experimental tasks conducted with the first child revealed that pronoun errors occurred in comprehension as well. Full control of the ME and YOU pronouns was not achieved until 25-27 months, around the same time when hearing children master these forms. Thus, the study provides evidence for a discontinuity in the child's transition from prelinguistic to linguistic communication. It is argued that aspects of linguistic structure and its acquisition appear to involve distinct, language-specific knowledge. (shrink)

ABSTRACT This essay examines Satyajit Ray’s children’s film Goopy Gyne Bagha Byne to interrogate the ways it signals the utopian. Contending that its utopian desire manifests through the trope of wish-fulfillment, it illuminates the trope’s centrality in the film’s imagination and articulation of freedom. Moreover, the essay suggests that the film’s utopianism is also anchored in its irrealist aesthetics. Engaging with the film’s figuration of ghosts and its violation of the narrative structure of the real, the essay illuminates how these (...) formal features inform, and are informed by, a utopian desire. Furthermore, the essay notes that despite its inherent utopianism, Goopy Gyne Bagha Byne fails to resolve some of the more obdurate social contradictions in its diegetic world, especially around questions of class, gender, and caste. Arguing that these “failures” are in keeping with its utopian spirit, I suggest they draw our attention to the imperfections of our real conditions of existence and flag the limits of textual utopianism. In so doing, they refer negatively to the social field beyond pointing to the abiding importance of class struggle as the ultimate agent of transformative social change. (shrink)

Early humans formed language units consisting of global and discrete dimensions of semiosis in dynamic opposition, or ‘growth points.’ At some point, gestures gained the power to orchestrate actions, manual and vocal, with significances other than those of the actions themselves, giving rise to cognition framed in dual terms. However, our proposal emphasizes natural selection of joint gesture-speech, not ‘gesture-first’ in language origin.

Orang-utans played a communication game in two studies testing their ability to produce and comprehend requestive pointing. While the ‘communicator’ could see but not obtain hidden food, the ‘donor’ could release the food to the communicator, but could not see its location for herself. They could coordinate successfully if the communicator pointed to the food, and if the donor comprehended his communicative goal and responded pro-socially. In Study 1, one orang-utan pointed regularly and accurately for peers. However, they responded (...) only rarely. In Study 2, a human experimenter played the communicator’s role in three conditions, testing the apes’ comprehension of points of different heights and different degrees of ostension. There was no effect of condition. However, across conditions one donor performed well individually, and as a group orang-utans’ comprehension performance tended towards significance. We explain this on the grounds that comprehension required inferences that they found difficult – but not impossible. The finding has valuable implications for our thinking about the development of pointing in phylogeny. (shrink)

The Philosophy of Gesture by Giovanni Maddalena is a multilayered volume: It is a "history of philosophy" book, endorsing a challenging anti-Kantian interpretation of Peirce and pragmatism. It is a "theoretical philosophy" book, dealing with classic issues—for example, the difference between synthetic and analytic, the definition of identity—and introducing a new concept, that of complete gesture. Finally, it is a book of "applied philosophy," pointing to a further application of the new concept of complete gesture to (...) the fields of pedagogy, morality, psychology, and literature.Dealing with all these three aspects in a single review would be impossible, maybe even unfair. Therefore, I will focus here mainly on the... (shrink)

Establishing and maintaining reference is a crucial part of discourse. In spoken languages, differential linguistic devices mark referents occurring in different referential contexts, that is, introduction, maintenance, and re-introduction contexts. Speakers using gestures as well as users of sign languages have also been shown to mark referents differentially depending on the referential context. This article investigates the modality-specific contribution of the visual modality in marking referential context by providing a direct comparison between sign language and co-speech gesture with speech (...) in elicited narratives. Across all forms of expression, we find that referents in subject position are referred to with more marking material in re-introduction contexts compared to maintenance contexts. Furthermore, we find that spatial modification is used as a modality-specific strategy in both DGS and German co-speech gesture, and that the configuration of referent locations in sign space and gesture space corresponds in an iconic and consistent way to the locations of referents in the narrated event. However, we find that spatial modification is used in different ways for marking re-introduction and maintenance contexts in DGS and German co-speech gesture. The findings are discussed in relation to the unique contribution of the visual modality to reference tracking in discourse when it is used in a unimodal system with full linguistic structure versus in a bimodal system that is a composite of speech and gesture. (shrink)

Human communication is by defaultpolysemiotic: it involves the spontaneous combination of two or moresemiotic systems, the most important ones beinglanguage,gesture, anddepiction. We formulate an original cognitive-semiotic framework for the analysis of polysemiosis, contrasting this with more familiar systems based on the ambiguous term “multimodality.” To be fully explicit, we developed a coding system for the analysis of polysemiotic utterances containing speech, gesture, and drawing, and implemented this in the ELAN video annotation software. We used this to analyze 23 (...) video-recordings of sand drawing performances on Paama, Vanuatu and 20 sand stories of the Pitjantjatjara culture in Central Australia. Methodologically we used theconceptual-empirical loopof cognitive semiotics: our theoretical framework guided general considerations, such as distinguishing between the “tiers” of gesture and depiction, and the three kinds of semiotic grounds (iconic, indexical, symbolic), but the precise decisions on how to operationalize these were made only after extensive work with the material. We describe the coding system in detail and provide illustrative examples from the Paamese and Pitjantjatjara data, remarking on both similarities and differences in the polysemiosis of the two cultural practices. We conclude by summarizing the contributions of the study and point to some directions for future research. (shrink)

Gestures and speech are clearly synchronized in many ways. However, previous studies have shown that the semantic similarity between gestures and speech breaks down as people approach transitions in understanding. Explanations for these gesture–speech mismatches, which focus on gestures and speech expressing different cognitive strategies, have been criticized for disregarding gestures’ and speech's integration and synchronization. In the current study, we applied three different perspectives to investigate gesture–speech synchronization in an easy and a difficult task: temporal alignment, semantic (...) similarity, and complexity matching. Participants engaged in a simple cognitive task and were assigned to either an easy or a difficult condition. We automatically measured pointing gestures, and we coded participant's speech, to determine the temporal alignment and semantic similarity between gestures and speech. Multifractal detrended fluctuation analysis was used to determine the extent of complexity matching between gestures and speech. We found that task difficulty indeed influenced gesture–speech synchronization in all three domains. We thereby extended the phenomenon of gesture–speech mismatches to difficult tasks in general. Furthermore, we investigated how temporal alignment, semantic similarity, and complexity matching were related in each condition, and how they predicted participants’ task performance. Our study illustrates how combining multiple perspectives, originating from different research areas (i.e., coordination dynamics, complexity science, cognitive psychology), provides novel understanding about cognitive concepts in general and about gesture–speech synchronization and task difficulty in particular. (shrink)

Tomasello and colleagues have offered various arguments to explain why apes find the comprehension of pointing difficult. They have argued that: (i) apes fail to understand communicative intentions; (ii) they fail to understand informative, cooperative communication, and (iii) they fail to track the common ground that pointing comprehension requires. In the course of a review of the literature on apes' production and comprehension of pointing, I reject (i) and (ii), and offer a qualified defence of (iii). Drawing (...) on work on expressive communication, I sketch an account of a mechanism by which ape gestural communication may proceed: the showing of expressive and naturally meaningful embodied behaviours. Such gestures are easily interpretable because they present rich evidence for a speaker's message. By contrast, pointing typically provides poor evidence for a speaker’s message, which must therefore be inferred from considerations in the interlocutors' common ground. This makes pointing comprehension comparatively difficult. (shrink)

Human motion gesture recognition is the most challenging research direction in the field of computer vision, and it is widely used in human-computer interaction, intelligent monitoring, virtual reality, human behaviour analysis, and other fields. This paper proposes a new type of deep convolutional generation confrontation network to recognize human motion pose. This method uses a deep convolutional stacked hourglass network to accurately extract the location of key joint points on the image. The generation and identification part of the network (...) is designed to encode the first hierarchy and the second hierarchy and show the spatial relationship of human body parts. The generator and the discriminator are designed as two parts in the network, and they are connected together in order to encode the possible relationship of appearance and, at the same time, the possibility of the existence of human body parts and the relationship between each part of the body and its parental part coding. In the image, the key nodes of the human body model and the general body posture can be identified more accurately. The method has been tested on different data sets. In most cases, the results obtained by the proposed method are better than those of other comparison methods. (shrink)

While recent studies have claimed that non-referential gestures are used to mark discourse-new and/or -accessible referents and focused information in adult speech, to our knowledge, no prior investigation has studied the relationship between information structure and gesture referentiality in children’s narrative speech from a developmental perspective. A longitudinal database consisting of 332 narratives performed by 83 children at two different time points in development was coded for IS and gesture referentiality. Results revealed that at both time points, both (...) referential and non-referential gestures were produced more with information that moves discourse forward and predication rather than topical or background information. Further, at 7–9 years of age, children tended to use more non-referential gestures to mark focus and comment constituents than referential gestures. In terms of the marking of the newness of discourse referents, non-referential gestures already seem to play a key role at 5–6 years old, whereas referential gestures did not show any patterns. This relationship was even stronger at 7–9 years old. All in all, our findings offer supporting evidence that in contrast with referential gestures, non-referential gestures have been found to play a key role in marking IS, and that the development of this relationship solidifies at a period in development that coincides with a spurt in non-referential gesture production. (shrink)

The article seeks a rapprochement between pragmatic and semantic theories of language by returning to a breaking point in the history of philosophy, the middle of the twentieth century, when these theoretical models began to evolve into distinct schools of thought. Philosophical accounts of this period explore various and intertwined dependencies between semantics and context; however, they only implicitly examine the potential of sounds and bodily gestures in bringing descriptive clarity to the modes and limits of such dependencies. The article (...) first investigates the way W. V. Quine conceptualized linguistic reference by combining behavioral models of language acquisition with more systematic explorations of syntax. It then turns to P. F. Strawson’s revisionary accounts of Kantian philosophy which reassess the silent strains of empiricism in Kant’s framework in order to identify possible grounds of reconciliation between pragmatic and semantic theories of language. Sound and gesture, two aspects that supplement language and give it an embodied feeling, are suggested as possible devices for formalizing intersections between context and semantic-oriented approaches. Poems by Wallace Stevens are used to think about the endurance of reference and how this endurance can find more compelling demonstration through an investigation of language as an embodied phenomenon. (shrink)

Resumen Al momento de comparar dos obras disímiles en términos de lo que aquí llamaremos el gesto lingüístico, pero similares en cuanto a su comprensión fundamental del decir respecto de su función, se vuelve clara la necesidad de expandir la metódica de acceso a tal fenómeno e incluir el momento analítico clave del ánimo del lenguaje. El presente análisis tiene como meta lograr tal expansión, desde el punto de partida ontológico que se propone en Ser y tiempo, para llevarlo hasta (...) un puerto -en apariencia- tan disímil como es la obra literaria de Roberto Bolaño.When comparing two works, that are dissimilar in terms of what we will call here the linguistic gesture, but similar in terms of their fundamental understanding of the concept with respect to its function, the need to expand the access to such phenomena becomes clear, and to also include the key analytical moment of the mood of the language. The purpose of this analysis is to achieve expansion, from the proposed ontological starting point in Being and time, to take it to a port -apparently- as different as the literary work of Roberto Bolaño. (shrink)

Infants initiate joint attentional exchanges by pointing relevant referents out to addressees. Over the second year of life, the functional meaning of the pointing gesture develops from declarative (sharing a referent) to informational (giving some information that is new to the addressee). This chapter analyzes this transition in the development of pointing based on experimental evidence about its production contexts and, in particular, of variables concerning the visibility of referents for infant and addressee. Further evidence is (...) reported concerning the association of pointing with vocalizations and visual checking towards the addressee in different temporal sequencing. It is argued that shared attention initiated by pointing plays a pivotal role in the intersection between social cognition and language development. (shrink)

As this foreword is written, at the dawn of the 21st Century, the cognitive sciences are in epistemological ferment. The exuberant optimism of the late 20th-Century adaptationist programme has, itself, started to fragment like the many fingers of a wave against the implacable shore of empirical reality. Dramatic new directions in the philosophy of mind and in developmental psychology are beginning to mature. This special issue, Pointing: Where Embodied Cognition meets the Symbolic Mind, edited by Massimiliano Cappuccio, brings together (...) in one volume an incendiary mix of the emerging generation of philosophers and language researchers, who bring their diverse perspectives to moor on one of the most fascinating phenomena in human development: the ability to co-orient in both space and time to a common focus with what seems, on the face of it, to be a simple pointing gesture. The emerging significance of research and analysis pertaining to pointing reflects a recent phase shift in the sciences concerned with mind and behaviour. To put these innovations in context, it might be worthwhile to review the sweeping upheavals that have recently occurred in the conceptual bedrocks of psychology and the philosophy of language. (shrink)

Did protolanguage users use discrete words that referred to objects, actions, locations, etc., and then, at some point, combine them; or on the contrary did they have words that globally indexed whole semantic complexes, and then come to divide them? Our answer is: early humans were forming language units consisting of global and discrete dimensions of semiosis in dynamic opposition. These units of thinking-for-speaking, or ‘growth points’ (GPs) were, jointly, analog imagery (visuo-spatio-motoric) and categorically-contrastive (-emic) linguistic encodings. This discrete-global duality (...) was a new mode of embodied cognition that enabled thinking and acting in new ways: the dawn of protolanguage. Where did this mode of cognition come from? We have some suggestions based on the hypothesis that gestures gained the power to orchestrate actions, manual and vocal, with significances other than those of the actions themselves, giving rise to cognition framed in the proposed dual terms. Note, however, our proposal is not one of the ‘gesture-first’ theories of language origins. Such theories predict what did not evolve: a language of pantomime; rather than what did evolve: an integrated system of synchronized gestures and spoken forms. GP theory is an account of the cognition underlying such an integrated system. A scenario for the evolutionary selection of this cognitive mode is ‘Mead’s Loop’, a model in which one’s cognition is enriched by one’s own gestures, insofar as they are objects in social interactions. (shrink)

Research has shown that gesture production supports learning across a number of tasks. It is unclear, however, whether gesture production during encoding can support narrative recall, who gesture production benefits most, and whether certain types of gestures are more beneficial than others. This study, therefore, investigated the effect of gesture production during the encoding of a narrative on subsequent narrative recall, and whether individuals’ levels of verbal and nonverbal memory moderated this effect. Additionally, this study investigated (...) whether producing certain types of gestures during encoding was more beneficial than others. Participants (N = 90, Mage = 20.43) read aloud a narrative while under instruction to produce gestures, under no specific instruction to produce gestures, or were required to keep their hands behind their back to prevent them from gesturing. Participants completed measures assessing verbal and nonverbal memory. While gesture production during encoding benefitted narrative recall (as measured through specific questions), verbal memory moderated the effect, such that gesture production was more beneficial for individuals with higher than lower verbal memory. Furthermore, producing representational gestures during encoding benefitted recall of points in the narrative at which those gestures were produced, while beat gestures had no effect. Findings have implications for understanding the mechanisms underlying the links between gesture and learning, as well as practical implications in instructional settings. (shrink)

This paper offers a biosemiotic account of the poetic aspects of gesture and speech in schizophrenia. The argument is that speech and gesture are not the mere expression of pre-verbal thoughts. Instead, meaning is enacted by the temporal and semantic coordination of speech and gesture. The bodily basis of language is highlighted by the fact that, failing to create language that is organized around topics, individuals with schizophrenia often rely on poetic associations in directing their utterances. Accordingly, (...) the analysis of schizophrenic speech and gesture based on McNeill’s Growth Point theory is enriched with reference to both Cowley’s (2007b) views on language as distributed social coordination, and Alexander’s (Biosemiotics 2(1):77–100, 2009) description of formal causes in nature. Schizophrenic language can be seen as a hypertrophied manifestation of the discourse-scaffolding function of poetic associations in speech and gesture. The analysis of schizophrenic language shows that language behavior need not be built on pre-verbal thoughts and important aspects of schizophrenic language, such as metalinguistic awareness and semiotic agency, can be clarified by applying concepts from biosemiotics. (shrink)

TThis paper aims to evidence the inherently metonymic nature of co-speech gestures. Arguing that motivation in gesture involves iconicity (similarity), indexicality (contiguity), and habit (conventionality) to varying degrees, it demonstrates how a set of metonymic principles may lend a certain systematicity to experientially grounded processes of gestural abstraction and enaction. Introducing visuo-kinetic signs as an umbrella term for co-speech gestures and signed languages, the paper shows how a frame-based approach to gesture may integrate different cognitive/functional linguistic and semiotic (...) accounts of metonymy (e.g., experiential domains, frame metonymy, contiguity, and pragmatic inferencing). The guiding assumption is that gestures metonymically profile deeply embodied, routinized aspects of familiar scenes, that is, the motivating context of frames. The discussion shows how gestures may evoke frame structures exhibiting varying degrees of groundedness, complexity, and schematicity: basic physical action and object frames; more complex frames; and highly abstract, complex frame structures. It thereby provides gestural evidence for the idea that metonymy is more basic and more directly experientially grounded than metaphor and thus often feeds into correlated metaphoric processes. Furthermore, the paper offers some initial insights into how metonymy also seems to induce the emergence of schematic patterns in gesture which may result from action-based and discourse-driven processes of habituation and conventionalization. It exemplifies how these forces may engender grammaticalization of a basic physical action into a gestural marker that shows strong metonymic form reduction, decreased transitivity, and interacting pragmatic functions. Finally, addressing basic metonymic operations in signed lexemes elucidates certain similarities regarding sign constitution in gesture and sign. English and German multimodal discourse data as well as German Sign Language (DGS) are drawn upon to illustrate the theoretical points of the paper. Overall, this paper presents a unified account of metonymy’s role in underpinning forms, functions, and patterns in visuo-kinetic signs. (shrink)

Name der Zeitschrift: Language and Cognition - An Interdisciplinary Journal of Language and Cognitive Science Jahrgang: 5 Heft: 4 Seiten: 313-343.

The central role of gestural language in Buddhism is widely acknowledged, as in the story of the Buddha pointing at the moon, the point being the student’s seeing beyond the finger to its gesture. Gesture’s role in dance is similarly central, as noted by scholars in the emerging interdisciplinary field of dance studies. Unsurprisingly, then, the intersection of these two fields is well-populated, including the formal gestures Buddhism inherited from classical Indian dance, and the masked dance of (...) the Mani Rimdu Festival. In this investigation, I will articulate a new Buddhist philosophy of gestural language, based on a new conception of emptiness that I locate in the work of contemporary U.S. choreographer Deborah Hay, as influenced by Nāgārjuna and Zen. And this, finally, suggests that contemporary Western philosophy should incorporate this compassion as a normative dimension to its own theorizing and practice. (shrink)