Preface This book is about semantics and logic. More specifically, it is about the semantics and logic of natural language; and, even more specifically than ...

How does sign language compare with gesture, on the one hand, and spoken language on the other? Sign was once viewed as nothing more than a system of pictorial gestures without linguistic structure. More recently, researchers have argued that sign is no different from spoken language, with all of the same linguistic structures. The pendulum is currently swinging back toward the view that sign is gestural, or at least has gestural components. The goal of this review is to elucidate the (...) relationships among sign language, gesture, and spoken language. We do so by taking a close look not only at how sign has been studied over the past 50 years, but also at how the spontaneous gestures that accompany speech have been studied. We conclude that signers gesture just as speakers do. Both produce imagistic gestures along with more categorical signs or words. Because at present it is difficult to tell where sign stops and gesture begins, we suggest that sign should not be compared with speech alone but should be compared with speech-plus-gesture. Although it might be easier to blur the distinction between sign and gesture, we argue that distinguishing between sign and gesture is essential to predict certain types of learning and allows us to understand the conditions under which gesture takes on properties of sign, and speech takes on properties of gesture. We end by calling for new technology that may help us better calibrate the borders between sign and gesture. (shrink)

In face-to-face interaction, speakers make multimodal contributions that exploit both the linguistic resources of spoken language and the visual and spatial affordances of gesture. In this paper, we argue that, in formulating and understanding such multimodal contributions, interlocutors apply the same principles of coherence that characterize the interpretation of natural language discourse. In particular, we use a close analysis of a series of naturally-occurring embodied discourses to argue for two key generalizations. First, communicators and their audiences draw on coherence relations (...) to establish interpretive connections between successive gestures and between gestures and speech. Second, coherence relations facilitate meaning-making by resolving the underspecified meaning of each communicative act through constrained inference over entities, propositions, and spatial frames made salient in the prior discourse. Our approach to gesture interpretation improves on previous work in better accounting for its flexibility, in capturing its constraints, and in laying the groundwork for formal and computational models. At the same time, it shows that gesture provides an important source of evidence to sharpen the theory of coherence relations and contextual resolution. (shrink)