The way we refer to things in the world is shaped by the immediate physical context as well as the discourse history. But what part of the discourse history is relevant to language use in the present? In four experiments, we combine the study of task‐based conversation with measures of recognition memory to examine the role of physical contextual cues that shape what speakers perceive to be a part of the relevant discourse history. Our studies leverage the differentiation effect, a (...) phenomenon in which speakers are more likely to use a modified expression to refer to an object (e.g., dotted sock) if they had previously described a similar object (e.g., striped sock) than when they had not described a similar object. Two physical cues—the background that framed the to‐be‐described pictures and the position of the pictures in the display—were manipulated to alter perceptions about the relevant discourse context. We measured the rate with which speakers modify referring expressions to differentiate current from past referents. Recognition memory measures following the conversation probed what was and was not remembered about past discourse referents and contexts. Analysis of modification rates indicated that these contextual grouping cues shaped perceptions about the relevant discourse context. The contextual cues did not affect memory for the referents, but the memory for past referents was better for speakers than for listeners. Our findings show that perceptions about the relevant discourse history are a key determinant of how language is used in the moment but also that conversational partners form asymmetric representations of the discourse history. (shrink)

We review major developmental evidence on the continuity from action to gesture to word and sign in human children, highlighting the important role of caregivers in the development of multimodal communication. In particular, the basic issues considered here and contributing to the current debate on the origins and development of the language-ready brain are: (1) links between early actions, gestures and words and similarities in representational strategies; (2) importance of multimodal communication and the interplay between gestures and spoken words; (3) (...) interconnections between early actions, gestures and signs. The innovation of this report is in connecting these themes together to relevant findings from studies on children between 6 and 36 months of age and highlighting interesting parallels in studies on ape communicative behavior. (shrink)

This paper first develops a theoretically motivated view of narrative as a special form of inferential, cooperative human communication, of the role that the past tense plays in the intersubjective coordination of narrators and readers, viz. that of ‘curtailing’ the immediate argumentative applicability of the represented situation, and of its relation to viewpoint management. In three case studies, it is subsequently shown how this helps to elucidate certain effects of present and past tense alternations in stories. While these effects are (...) multi-faceted and highly text-specific, there is a common denominator of the use of the past tense in the dimension of narrator-reader communication in the narratives. The analysis supports an independently motivated conception of intersubjectivity that assigns a special status to ‘coordination with other minds’, apart from senders and addressees. (shrink)

Differences in language use and structures between signed and spoken languages have often been attributed to so-called language “modality.” Indeed, this is derived from the conception that spoken languages resort to both the oral-aural channel of speech and the visual-kinesic channel of visible bodily action whereas signed languages only resort to the latter. This paper addresses the use of enactment, a depictive communicative strategy whereby language users imitate referents in signed and spoken languages. Reviewing comparative research on enactment, this paper (...) highlights theoretical and methodological shortcomings in prior works. First, a broader set of causal explanations needs to be taken into account when interpreting differences between signing and speaking communities. A more comprehensive conceptual toolbox ensures that differences are not automatically attributed to modality. In particular, less-studied factors of language diversity, like sociolinguistic and cultural ecologies, and how they interact with other factors should be considered. Second, diversity in enactment across signed and spoken languages is shown to be inadequately and insufficiently documented. It is argued that by comparing enactment across more diverse signing and speaking communities and using large, directly comparable corpora, solid analyses can be carried out, enabling a better understanding of how and why different communities use enactment in similar or different ways. (shrink)

This study examines the form and function of gestural depictions that develop over extended stretches of concept explanation by a philosopher. Building onStreeck’s (2009)explorations of depiction by gesture, we examine how this speaker’s process of exposition involves sequences of multimodal, analogical depiction by which the philosophical concepts are not only expressed through gesture forms, but also dynamically analyzed and construed through gestural activity. Drawing on perspectives of gesture as active meaning making (Müller 2014,2016,Streeck 2009), we argue that the build-up of (...) gestures in depiction sequences, activated through a multimodal metaphor (Müller & Cienki 2009), engages the wider philosophical standpoint of the speaker. Using video analysis supported by interview data, we demonstrate how examination of gestures within and across discourse can lead to understanding of how dynamic, embodied, and subjective processes of conceptualization contribute to philosophical theorizing. (shrink)

This article examines how therapists and patients start building and managing relationships and pursue institutional goals at the same time. Based on a corpus of 6 audio-recorded therapies (client-centered therapy and psychodynamic therapy), I investigate first encounters between therapists and patients as the starting points of any therapeutical process and the place where a relationship between the interactants is established for the first time. Following a microlinguistic qualitative approach and applying methods from conversation analysis and discourse analysis, I show how (...) therapists, on the one hand, try to align with patients to build a positive working alliance and, on the other hand, work to fulfill specific interactive tasks of therapeutic discourse which demand disaligning with the patients’ communicative activity and their interactive expectations. Specific interactive “jobs” that need to be fulfilled in psychotherapy are identified, namely the performance of institutional roles by the interactants, the establishment of an interaction structure and the pursuit of helpful change in the patient. I show at which places in the interaction therapists (dis-)align with the patients’ projected communicative activity and how aligning and disaligning are related to the interactive process and the establishment and performance of these interactive jobs. The analysis shows that, at the beginning of therapy, alignment and disalignment are both important processes for the following reasons: Aligning with the patient contributes to a positive relationship, which has been shown to be vital for successful psychotherapy, while disaligning introduces the patient to the specific discursive mechanisms that characterize therapeutic discourse and constitute the basis for its effectiveness. Overall, the paper argues that reducing therapy to a dichotomy between relationship and “technique” seems overly simplistic, as both aspects need to be handled and managed at the same time. (shrink)

Peter Hanks and Scott Soames both defend pragmatic solutions to the problem of the unity of the proposition. According to them, what ties together Tim and baldness in the singular proposition expressed by ‘Tim is bald’ is an act of the speaker : the act of predicating baldness of Tim. But Soames construes that act as force neutral and noncommittal while, for Hanks, it is inherently assertive and committal. Hanks answers the Frege–Geach challenge by arguing that, in complex sentences, the (...) force inherent in the content of an embedded sentence is cancelled. Indrek Reiland has recently objected to Hanks’s proposal that it faces a dilemma: either force cancellation dissolves the unity of the proposition secured by the cancelled act of assertion, or Hanks’s proposal reduces to Soames’s. In this paper, I respond to Reiland by offering an analysis of force cancellation which gets rid of the alleged dilemma. The proposal is based on a set of distinctions from speech act theory : between two senses of ’force’, two types of act, and two types of context. The role of simulation in force cancellation is emphasized, and connections drawn to broader issues such as the evolution of complex language. (shrink)

Gesture-first theories of language origins often raise two unsubstantiated arguments against vocal origins. First, they argue that great ape vocal behavior is highly constrained, limited to a fixed, species-typical repertoire of reflexive calls. Second, they argue that vocalizations lack any significant potential to ground meaning through iconicity, or resemblance between form and meaning. This paper reviews the considerable evidence that debunks these two “myths”. Accumulating evidence shows that the great apes exercise voluntary control over their vocal behavior, including their breathing (...) apparatus, larynx, and supralaryngeal articulators. They are also able to learn new vocal behaviors, and even show some rudimentary ability for vocal imitation. In addition, an abundance of research demonstrates that the vocal modality affords rich potential for iconicity. People can understand iconicity in sound symbolism, and they can produce iconic vocalizations to communicate a diverse range of meanings. Thus, two of the primary arguments against vocal origins theories are not tenable. As an alternative, the paper concludes that the origins of language – going as far back as our last common ancestor with great apes – are rooted in iconicity in both gesture and vocalization. (shrink)

Theater rehearsals are (usually) confronted with the problem of having to transform a written text into an audio-visual, situated and temporal performance. Our contribution focuses on the emergence and stabilization of a gestural form as a solution for embodying a certain aesthetic concept which is derived from the script. This process involves instructions and negotiations, making the process of stabilization publicly and thus intersubjectively accessible. As scenes are repeatedly rehearsed, rehearsals are perspicuous settings for tracking interactional histories. Based on videotaped (...) professional theatre interactions in Germany, we focus on consecutive instances of rehearsing the same scene and trace the interactional history of a particular gesture. This gesture is used by the director to instruct the actors to play a particular aspect of a scene adopting a certain aesthetic concept. Stabilization requires the emergence of shared knowledge. We will show the practices by which shared knowledge is established over time during the rehearsal process and, in turn, how the accumulation of knowledge contributes to a change in the interactional practices themselves. Specifically, we show how a gesture emerges in the process of developing and embodying an aesthetic concept, and how this gesture eventually becomes a sign that refers to and evokes accumulated knowledge. At the same time, we show how this accumulated knowledge changes the instructional activities in the rehearsal process. Our study contributes to the overall understanding of knowledge accumulation in interaction in general and in theater rehearsals in particular. At the same time, it is devoted to the central importance of gestures in theater, which are both a means and a product of theatrical staging. (shrink)

The goal of the article is to offer a framework against which relations between gesture and sign can be systematically explored beyond the current literature. It does so by a) reconstructing the history of the discussion in the field of gesture studies, focusing on three leading positions (Kendon, McNeill, Goldin-Meadow); and b) by formulating a position to illustrate how this can be achieved. The paper concludes by emphasizing the need for systematic cross-linguistic research on multimodal use of language in its (...) signed and spoken forms. (shrink)

Journal Name: Cognitive Linguistics Issue: Ahead of print.

Pantomime has long been considered distinct from co-speech gesture. It has therefore been argued that pantomime cannot be part of gesture-speech integration. We examine pantomime as distinct from silent gesture, focusing on non-co-speech gestures that occur in the midst of children’s spoken narratives. We propose that gestures with features of pantomime are an infrequent but meaningful component of a multimodal communicative strategy. We examined spontaneous non-co-speech representational gesture production in the narratives of 30 monolingual English-speaking children between the ages of (...) 8- and 11-years. We compared the use of co-speech and non-co-speech gestures in both autobiographical and fictional narratives and examined viewpoint and the use of non-manual articulators, as well as the length of responses and narrative quality. The use of non-co-speech gestures was associated with longer narratives of equal or higher quality than those using only co-speech gestures. Non-co-speech gestures were most likely to adopt character-viewpoint and use non-manual articulators. The present study supports a deeper understanding of the term pantomime and its multimodal use by children in the integration of speech and gesture. (shrink)

Journal Name: Cognitive Linguistics Issue: Ahead of print.

The framework of depicting put forward byClark (2016)offers a schematic vantage point from which to examine iconic language use. Confronting the framework with empirical data, we consider some of its key theoretical notions. Crucially, by reconceptualizing the typology of depictions, we identify an overlooked domain in the literature: “speech-embedded nonverbal depictions,” namely cases where meaning is communicated iconically, nonverbally, and without simultaneously co-occurring speech. In addition to contextualizing the phenomenon in relation to existing research, we demonstrate, with examples from American (...) TV talk shows, how such depictions function in real-life language use, offering a brief sketch of their complexities and arguing also for their theoretical significance. (shrink)

Investigations of iconicity in language, whereby interactants coordinate meaningful bodily actions to create resemblances, are prevalent across the human communication sciences. However, when it comes to analysing and comparing iconicity across different interactions and modes of communication, it is not always clear we are looking at the same thing. For example, tokens of spoken ideophones and manual depicting actions may both be analysed as iconic forms. Yet spoken ideophones may signal depictive and descriptive qualities via speech, while manual actions may (...) signal depictive, descriptive, and indexical qualities via the shape, movement, and placement of the hands in space. Furthermore, each may co-occur with other semiotics articulated with the face, hands, and body within composite utterances. The paradigm of iconicity as a single property is too broad and coarse for comparative semiotics, as important details necessary for understanding the range of human communicative potentialities may be masked. Here, we draw on semiotic approaches to language and communication, including the model of language as signalled via describing, indicating and/or depicting and the notion of non-referential indexicality, to illustrate the multidimensionality of iconicity in co-present interactions. This builds on our earlier proposal for analysing how different methods of semiotic signalling are combined in multimodal language use. We discuss some implications for the language and communication sciences and explain how this approach may inform a theory of biosemiotics. (shrink)

Using multimodal conversation analysis, we investigate how novices learning the “inner body” acting technique in the context of a community theater project share their experiences of the bodily exercises through verbal and embodied conduct. We focus on how verbal description and bodily enactment of the experience mutually elaborate each other, and how the experienced sensorimotor and affective qualities are made to be witnessed and recognized by the others. Participants describe their experiences without naming qualities. Instead, a display of the experienced (...) qualities is made accessible to others through coordinating the unfolding talk and bodily conduct. In particular, we show how grammatical and action projection is fulfilled by interconnected verbal and embodied conduct, with body movement and posture giving off ineffable experiential qualities. The moving body appears both as a source of the experience and as a resource for depicting perceived qualities to others; additional resources (non-specific person reference and gaze aversion) contribute to organizing the subjective and intersubjective layers of the reflection of the experiences. The study contributes to and extends recent research on sensoriality in interaction by focusing on phenomena of proprioception and interoception. The data are two cases drawn from 60 h of video-recordings made in the context of a devised community theater project. The data are in Finnish with English translations. (shrink)

Analogy researchers do not often examine gesture, and gesture researchers do not often borrow ideas from the study of analogy. One borrowable idea from the world of analogy is the importance of distinguishing between attributes and relations. Gentner observed that some metaphors highlight attributes and others highlight relations, and called the latter analogies. Mirroring this logic, we observe that some metaphoric gestures represent attributes and others represent relations, and propose to call the latter analogical gestures. We provide examples of such (...) analogical gestures and show how they relate to the categories of iconic and metaphoric gestures described previously. Analogical gestures represent different types of relations and different degrees of relational complexity, and sometimes cohere into larger analogical models. Treating analogical gestures as a distinct phenomenon prompts new questions and predictions, and illustrates one way that the study of gesture and the study of analogy can be mutually informative. (shrink)

Social robots serve people as tutors, caretakers, receptionists, companions, and other social agents. People know that the robots are mechanical artifacts, yet they interact with them as if they were actual agents. How is this possible? The proposal here is that people construe social robots not as social agents per se, but as depictions of social agents. They interpret them much as they interpret ventriloquist dummies, hand puppets, virtual assistants, and other interactive depictions of people and animals. Depictions as a (...) class consist of three physical scenes with part-by-part mappings between them: (a) a base scene (the raw physical artifact), (b) the depiction proper (the artifact construed as a depiction), and (c) the scene depicted (the scene people are to imagine). With social robots, evidence shows people form the same three scenes plus mappings: They perceive the raw machinery of a robot, construe it as a depiction of a character, and, using the depiction as a guide, engage in the pretense that they are interacting with the character depicted. With social robots, people also recognize three classes of agents – the characters depicted, the intended recipients of the depictions (those who view or interact with the robots), and the authorities responsible for the robots (the designers, makers, and owners). Construing social robots as depictions, we argue, accounts for many phenomena not covered by alternative models. (shrink)

We take up issues raised in the commentaries about our proposal that social robots are depictions of social agents. Among these issues are the realism of social agents, experiencing robots, communicating with robots, anthropomorphism, and attributing traits to robots. We end with comments about the future of social robots.

The article explores cross-modal iconic relations in nine diverse Western-music songs ranging from 1600 to 2015, all of them thematizing dysphoric weeping. Initial input comes from five recurrent features observed in ancient Greek texts associated with performative events, including the prominence of sound, interjections and strong self-referentiality, repetitions and refrains, the motif of endlessness, and tears associated with streams of water, dew, and libation liquids. The analysis adopts Peirce’s conceptual distinction between image, diagram, and metaphor iconicity, although the continuum reading (...) proposed by several scholars is ultimately favored. The sample turns out to offer plenty of evidence of iconic relations. Interjections, falsetto, verbal repetitions, musical repetitions, musical rendering of sighs and endlessness, ostinato patterns, downward notes, and water images (rain and rivers), all of this is shown to substantiate cross-modal iconicity encompassing weeping, lyrics, music (including musical notation), and images, in different combinations. Quantitative investigations confirming/disconfirming and adding iconicity patterns, and comparative analyses linking Western and non-Western lament traditions constitute desiderata for future work. (shrink)

Already at a relatively early stage, modern sign language linguistics focused on the representation of (actions, locations, and motions of) referents (1) through the use of the body and its different articulators and (2) through the use of particular handshapes (in combination with an orientation, location, and/or movement). Early terminology for (1) includesrole playing, role shifting, androle takingand for (2)classifier constructions/predicatesandverbs of motion and location. More recently, however, new terms, includingenactmentandconstructed actionfor (1) anddepicting signsfor (2) have been introduced. This article (...) provides a brief overview of the history of enactment and depiction in the sign linguistic literature but mainly focuses on issues related to terminology (and terminology shifts). First, we consider the relation between role shifting and constructed action. We question the idea that these terms can be used interchangeably and rather suggest that they capture different, but related functions. Subsequently, we zoom in on the conceptualization of depicting signs, indicating verbs, pointing signs and fully lexical signs and the relation between these signs and the method of depicting. Where earlier research often associates depicting with the use of specific types of structures, we promote the idea that depicting is a semiotic diverse practice. In doing so, we show that the conceptualization of the different sign types and the terms that are used to refer to these phenomena do not accurately capture the way these signs are used in actual signed discourse and propose a reconceptualization of the different sign types in the lexico-grammar of Flemish Sign Language (VGT) as composite signs that can describe, depict and indicate meaning in various ways. In this way, this article illustrates (1) the risks that may come with the execution of terminology shifts and (2) the importance of making a clear distinction between form and function, i.e., we show that it is important to be careful with assuming a (too) exclusive relation between a certain function and one or more particular forms. (shrink)