Alignment in Multimodal Interaction: An Integrative Framework

Cognitive Science 44 (11):e12911 (2020)
  Copy   BIBTEX

Abstract

When people are engaged in social interaction, they can repeat aspects of each other’s communicative behavior, such as words or gestures. This kind of behavioral alignment has been studied across a wide range of disciplines and has been accounted for by diverging theories. In this paper, we review various operationalizations of lexical and gestural alignment. We reveal that scholars have fundamentally different takes on when and how behavior is considered to be aligned, which makes it difficult to compare findings and draw uniform conclusions. Furthermore, we show that scholars tend to focus on one particular dimension of alignment (traditionally, whether two instances of behavior overlap in form), while other dimensions remain understudied. This hampers theory testing and building, which requires a well‐defined account of the factors that are central to or might enhance alignment. To capture the complex nature of alignment, we identify five key dimensions to formalize the relationship between any pair of behavior: time, sequence, meaning, form, and modality. We show how assumptions regarding the underlying mechanism of alignment (placed along the continuum of priming vs. grounding) pattern together with operationalizations in terms of the five dimensions. This integrative framework can help researchers in the field of alignment and related phenomena (including behavior matching, mimicry, entrainment, and accommodation) to formulate their hypotheses and operationalizations in a more transparent and systematic manner. The framework also enables us to discover unexplored research avenues and derive new hypotheses regarding alignment.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 90,593

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Chapter 9. Image Alignment in Multimodal Metaphor.Norman Y. Teng - 2009 - In Eduardo Urios-Aparisi & Charles J. Forceville (eds.), Multimodal Metaphor. Berlin, New York: Mouton de Gruyter. pp. 197–212.
Multimodal logic for syntax.Alain Lecomte - 1998 - Logica Trianguli 2:49-72.
Carving language for social coordination.Riccardo Fusaroli & Kristian Tylén - 2012 - Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systemsinteraction Studies / Social Behaviour and Communication in Biological and Artificial Systemsinteraction Studies 13 (1):103-124.

Analytics

Added to PP
2020-10-30

Downloads
17 (#742,076)

6 months
5 (#246,492)

Historical graph of downloads
How can I increase my downloads?

Author's Profile