Knowledge of the complexity of human communication comes from three main sources – (i) studies of the linguistics and neuropsychology of dysfunction after brain injury; (ii) studies of the development of social communication in infancy, and its dysfunction in developmental psychopathologies; and (iii) the evolutionary history of human communicative interaction. Together, these suggest the need for a broad, integrated theory of communication of which language forms a small but critical component.

The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have (...) limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the nonlinear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique’s application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a “Programming Basics” module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data. (shrink)

Complex work in teams requires coordination across team members and their technology as well as the ability to change and adapt over time to achieve effective performance. To support such complex interactions, recent efforts have worked toward the design of adaptive human-autonomy teaming systems that can provide feedback in or near real time to achieve the desired individual or team results. However, while significant advancements have been made to better model and understand the dynamics of team interaction and its relationship (...) with task performance, appropriate measures of team coordination and computational methods to detect changes in coordination have not yet been widely investigated. Having the capacity to measure coordination in real time is quite promising as it provides the opportunity to provide adaptive feedback that may influence and regulate teams’ coordination patterns and, ultimately, drive effective team performance. A critical requirement to reach this potential is having the theoretical and empirical foundation from which to do so. Therefore, the first goal of the paper is to review approaches to coordination dynamics, identify current research gaps, and draw insights from other areas, such as social interaction, relationship science, and psychotherapy. The second goal is to collate extant work on feedback and advance ideas for adaptive feedback systems that have potential to influence coordination in a way that can enhance the effectiveness of team interactions. In addressing these two goals, this work lays the foundation as well as plans for the future of human-autonomy teams that augment team interactions using coordination-based measures. (shrink)

Collaborators generally coordinate their activities through communication, during which they readily negotiate a shared lexicon for activity-related objects. This social-pragmatic activity both recruits and affects cognitive and social-cognitive processes ranging from selective attention to perspective taking. We ask whether negotiating reference also facilitates category learning or might private verbalization yield comparable facilitation? Participants in three referential conditions learned to classify imaginary creatures according to combinations of functional features—nutritive and destructive—that implicitly defined four categories. Remote partners communicated in the Dialogue condition. (...) In the Monologue condition, participants recorded audio descriptions for their own later use. Controls worked silently. Dialogue yielded better category learning, with wider distribution of attention. Monologue offered no benefits over working silently. We conclude that negotiating reference compels collaborators to find communicable structure in their shared activity; this social-pragmatic constraint accelerates category learning and likely provides much of the benefit recently ascribed to learning labeled categories. (shrink)

How is linguistic communication possible? How do we come to share the same meanings of words and utterances? One classical position holds that human beings share a transcendental “platonic” ideality independent of individual cognition and language use (Frege 1948). Another stresses immanent linguistic relations (Saussure 1959), and yet another basic embodied structures as the ground for invariant aspects of meaning (Lakoff and Johnson 1999). Here we propose an alternative account in which the possibility for sharing meaning is motivated by four (...) sources of structural stability: 1) the physical constraints and affordances of our surrounding material environment, 2) biological constraints of our human bodies, 3) social normative constraints of culture and society, and 4) the local history of social interactions. These structures and constraints interact in dynamical ways in actual language usage situations: local dialogical and social dynamics motivate and stabilize the profiling of a conceptual space already highly structured by our shared biology, culture, and environment. We will substantiate this perspective with reference to recent studies in experimental pragmatics and semiotics in which participants interact linguistically to solve cooperative tasks. Three main cases will be considered: The dynamic grounding of linguistic categories, the construction of conceptual models to relate entities in a scene, and the construction of shared conceptual scales for assessing and appraising subjective experiences. (shrink)

People coordinate body postures and gaze patterns during conversation. We review literature showing that (1) action embodies cognition, (2) postural coordination emerges spontaneously when two people converse, (3) gaze patterns influence postural coordination, (4) gaze coordination is a function of common ground knowledge and visual information that conversants believe they share, and (5) gaze coordination is causally related to mutual understanding. We then consider how coordination, generally, can be understood as temporarily coupled neuromuscular components that function as a collective unit (...) known as a coordinative structure in the motor control literature. We speculate that the coordination of gaze and body sway found in conversation may be understood as a cross‐person coordinative structure that embodies the goals of the joint action system. (shrink)

In recent years, a number of philosophers and cognitive scientists have advocated for an ‘interactive turn’ in the methodology of social-cognition research: to become more ecologically valid, we must design experiments that are interactive, rather than merely observational. While the practical aim of improving ecological validity in the study of social cognition is laudable, we think that the notion of ‘interaction’ is not suitable for this task: as it is currently deployed in the social cognition literature, this notion leads to (...) serious conceptual and methodological confusion. In this paper, we tackle this confusion on three fronts: 1) we revise the ‘interactionist’ definition of interaction; 2) we demonstrate a number of potential methodological confounds that arise in interactive experimental designs; and 3) we show that ersatz interactivity works just as well as the real thing. We conclude that the notion of ‘interaction’, as it is currently being deployed in this literature, obscures an accurate understanding of human social cognition. (shrink)

This paper outlines a first attempt to model the special constraints that arise in language processing in conversation, and to explore the implications such functional considerations may have on language typology and language change. In particular, we focus on processing pressures imposed by conversational turn-taking and their consequences for the cultural evolution of the structural properties of language. We present an agent-based model of cultural evolution where agents take turns at talk in conversation. When the start of planning for the (...) next turn is constrained by the position of the verb, the stable distribution of dominant word orders across languages evolves to match the actual distribution reasonably well. We suggest that the interface of cognition and interaction should be a more central part of the story of language evolution. (shrink)

Conversation is supported by the beliefs that people have in common and the perceptual experience that they share. The visual context of a conversation has two aspects: the information that is available to each conversant, and their beliefs about what is present for each other. In our experiment, we separated these factors for the first time and examined their impact on a spontaneous conversation. We manipulated the fact that a visual scene was shared or not and the belief that a (...) visual scene was shared or not. Participants watched videos of actors talking about a controversial topic, then discussed their own views while looking at either a blank screen or the actors. Each believed (correctly or not) that their partner was either looking at a blank screen or the same images. We recorded conversants’ eye movements, quantified how they were coordinated, and analyzed their speech patterns. Gaze coordination has been shown to be causally related to the knowledge people share before a conversation, and the information they later recall. Here, we found that both the presence of the visual scene, and beliefs about its presence for another, influenced language use and gaze coordination. (shrink)

Communicative interactions across different species have so far received relatively little attention from cognitive or behavioral scientists. Most research in this area views the process of communication as the adaptive interaction of manipulative signalers and information-assessing receivers. This paper discusses some shortcomings of the information/influence model of communication, particularly in the empirical study of interspecific communicative interactions. It then presents an alternative theoretical model, based on recent contributions in psycholinguistics and semiotics. The semiotic alignment model views communication as a dynamic (...) process of joint semiosis resulting in the alignment of the interactants’ own-worlds (Umwelten). It is argued that this model can improve our understanding of communicative interactions between heterospecifics and provide the basis for future work in the empirical study of interspecific communication. (shrink)

Communicative interactions across different species have so far received relatively little attention from cognitive or behavioral scientists. Most research in this area views the process of communication as the adaptive interaction of manipulative signalers and information-assessing receivers. This paper discusses some shortcomings of the information/influence model of communication, particularly in the empirical study of interspecific communicative interactions. It then presents an alternative theoretical model, based on recent contributions in psycholinguistics and semiotics. The semiotic alignment model views communication as a dynamic (...) process of joint semiosis resulting in the alignment of the interactants’ own-worlds (Umwelten). It is argued that this model can improve our understanding of communicative interactions between heterospecifics and provide the basis for future work in the empirical study of interspecific communication. (shrink)

Our target article proposed that language production and comprehension are interwoven, with speakers making predictions of their own utterances and comprehenders making predictions of other people's utterances at different linguistic levels. Here, we respond to comments about such issues as cognitive architecture and its neural basis, learning and development, monitoring, the nature of forward models, communicative intentions, and dialogue.

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume (...) that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation. We show how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, how they interweave production and comprehension processes, and how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal. (shrink)

Why is interaction so simple? This article presents a theory of interaction based on the use of shared representations as “coordination tools” (e.g., roundabouts that facilitate coordination of drivers). By aligning their representations (intentionally or unintentionally), interacting agents help one another to solve interaction problems in that they remain predictable, and offer cues for action selection and goal monitoring. We illustrate how this strategy works in a joint task (building together a tower of bricks) and discuss its requirements from a (...) computational viewpoint. (shrink)

Symbols enable people to organize and communicate about the world. However, the ways in which symbolic knowledge is learned and then represented in the mind are poorly understood. We present a formal analysis of symbolic learning—in particular, word learning—in terms of prediction and cue competition, and we consider two possible ways in which symbols might be learned: by learning to predict a label from the features of objects and events in the world, and by learning to predict features from a (...) label. This analysis predicts significant differences in symbolic learning depending on the sequencing of objects and labels. We report a computational simulation and two human experiments that confirm these differences, revealing the existence of Feature-Label-Ordering effects in learning. Discrimination learning is facilitated when objects predict labels, but not when labels predict objects. Our results and analysis suggest that the semantic categories people use to understand and communicate about the world can only be learned if labels are predicted from objects. We discuss the implications of this for our understanding of the nature of language and symbolic thought, and in particular, for theories of reference. (shrink)

The pull to coordinate with other individuals is fundamental, serving as the basis for our social connectedness to others. Discussed is a dynamical and ecological perspective to joint action, an approach that embeds the individual’s mind in a body and the body in a niche, a physical and social environment. Research on uninstructed coordination of simple incidental rhythmic movement, along with research on goal‐directed, embodied cooperation, is reviewed. Finally, recent research is discussed that extends the coordination and cooperation studies, examining (...) how synchronizing with another, and how emergent social units of perceiving and acting are reflected in people’s feelings of connection to others. (shrink)

A variety of theoretical frameworks predict the resemblance of behaviors between two people engaged in communication, in the form of coordination, mimicry, or alignment. However, little is known about the time course of the behavior matching, even though there is evidence that dyads synchronize oscillatory motions (e.g., postural sway). This study examined the temporal structure of nonoscillatory actions—language, facial, and gestural behaviors—produced during a route communication task. The focus was the temporal relationship between matching behaviors in the interlocutors (e.g., facial (...) behavior in one interlocutor vs. the same facial behavior in the other interlocutor). Cross-recurrence analysis revealed that within each category tested (language, facial, gestural), interlocutors synchronized matching behaviors, at temporal lags short enough to provide imitation of one interlocutor by the other, from one conversational turn to the next. Both social and cognitive variables predicted the degree of temporal organization. These findings suggest that the temporal structure of matching behaviors provides low-level and low-cost resources for human interaction. (shrink)

During dialog, references are presented, accepted, and potentially reused. Two experiments were conducted to examine reuse in a naturalistic setting. In Experiment 1, where the participants interacted face to face, self-presented references and references accepted through verbatim repetition were reused more. Such biases persisted after the end of the interaction. In Experiment 2, where the participants interacted over the phone, reference reuse mainly depended on whether the participant could see the landmarks being referred to, although this bias seemed to be (...) only transient. Consistent with the memory-based approach to dialog, these results shed light on how differences in accessibility in memory affect the unfolding of the interaction. (shrink)

Healey et al. introduce the special issue with a brief overview of work on communication in the Cognitive Sciences and some of the historical and conceptual influences that have marginalized the study of miscommunication. Drawing on more recent work in Cognitive Science and Conversation Analysis they argue that miscommunication is in fact a highly structured, ubiquitous phenomenon that is fundamental to human interaction.

Human social coordination is often mediated by language. Through verbal dialogue, people direct each other’s attention to properties of their shared environment, they discuss how to jointly solve problems, share their introspections, and distribute roles and assignments. In this article, we propose a dynamical framework for the study of the coordinative role of language. Based on a review of a number of recent experimental studies, we argue that shared symbolic patterns emerge and stabilize through a process of local reciprocal linguistic (...) alignment. Such patterns in turn come to facilitate and refine social coordination by enabling the alignment, joint construction and navigation of conceptual models and actions. Implications of the framework are illustrated and discussed in relation to a case study where dyads of interlocutors interact verbally to reach joint decisions in a perceptual discrimination task. (shrink)

Coordination is a widely employed term across recent quantitative and qualitative approaches to intersubjectivity, particularly approaches that give embodiment and enaction central explanatory roles. With a focus on linguistic and bodily coordination in conversational contexts, I review the operational meaning of coordination in recent empirical research and related theorizing of embodied intersubjectivity. This discussion articulates what must be involved in treating linguistic meaning as dynamic processes of coordination. The coordination approach presents languaging as a set of dynamic self-organizing processes and (...) actions on multiple timescales and across multiple modalities that come about and work in certain domains (those jointly constructed in social, interactive, high-order sense-making). These processes go beyond meaning at the level that is available to first-person experience. I take one crucial consequence of this to be the ubiquitously moral nature of languaging with others. Languaging coordinates experience, among other levels of behavior and event. Ethical effort is called for by the automatic autonomy-influencing forces of languaging as coordination. (shrink)

Coordination between speakers in dialogue requires balancing repetition and change, the old and the new. Interlocutors tend to reuse established forms, relying on communicative precedents. Yet linguistic interaction also necessitates adaptation to changing contexts or dynamic tasks, which might favor abandoning existing precedents in favor of better communicative alternatives. We explored this tension using a maze game task in which individual participants and interacting pairs had to describe figures and their positions in one of two possible maze types: a regular (...) maze, in which the grid‐like structure of the maze is highlighted, and an irregular maze, in which specific parts of the maze are salient. Participants repeated this task several times. Both individuals and interacting pairs were affected by the different maze layouts, initially using more idiosyncratic description schemes for irregular mazes and more systematic schemes for regular mazes. Interacting pairs, but not individuals, abandoned their unsystematic initial descriptions in favor of a more systematic approach, which was better adapted for repeated interaction. Our results show communicative conventions are initially shaped by context, but interaction opens up the possibility for change if better alternatives are available. (shrink)

It is well established that language production and comprehension are influenced by information status, for example, whether information is given, new, topical, or predictable, and many scholars suggest that an important component of information status is keeping track of what information is in common ground, and what is not. Information status affects both speakers' choices and how listeners interpret the speaker's meaning. Although there is a wealth of scholarly work on information status, there is no consensus on the mechanisms by (...) which it is used, and in fact relatively little discussion of the underlying representations and psycholinguistic mechanisms. Moreover, a major challenge to understanding information status is that its effects are notoriously variable. This study considers existing proposals about information status, focusing on two questions: how is it represented; and by what mechanisms is it used? I propose that it is important to consider whether representations and mechanisms can be classified as either explicit or emergent. Based on a review of existing evidence, I argue that information status representations are most likely emergent, but the mechanisms by which they are used are both explicit and emergent. This review provides one of the first considerations of information status processing across multiple domains. (shrink)