What is the role of language in social interaction? What does language bring to social encounters? We argue that language can be conceived of as a tool for interacting minds, enabling especially effective and flexible forms of social coordination, perspective-taking and joint action. In a review of evidence from a broad range of disciplines, we pursue elaborations of the language-as-a-tool metaphor, exploring four ways in which language is employed in facilitation of social interaction. We argue that language dramatically extends the (...) possibility-space for interaction, facilitates the profiling and navigation of joint attentional scenes, enables the sharing of situation models and action plans, and mediates the cultural shaping of interacting minds. (shrink)

A key emergent property of group social dynamic is synchrony–the coordination of actions, emotions, or physiological processes between group members. Despite this fact and the inherent nested structure of groups, little research has assessed physiological synchronization between group members from a multi-level perspective, thus limiting a full understanding of the dynamics between members. To address this gap of knowledge we re-analyzed a large dataset comprising physiological and psychological data that were collected in two laboratory studies that involved two different social (...) group tasks. In both studies, following the group task, members reported their experience of group cohesion via questionnaires. We utilized a non-linear analysis method-multidimensional recurrence quantification analysis that allowed us to represent physiological synchronization in cardiological interbeat intervals between group members at the individual-level and at the group-level. We found that across studies and their conditions, the change in physiological synchrony from baseline to group interaction predicted a psychological sense of group cohesion. This result was evident both at the individual and the group levels and was not modified by the context of the interaction. The individual- and group-level effects were highly correlated. These results indicate that the relationship between synchrony and cohesion is a multilayered construct. We re-affirm the role of physiological synchrony for cohesion in groups. Future studies are needed to crystallize our understanding of the differences and similarities between synchrony at the individual-level and synchrony at the group level to illuminate under which conditions one of these levels has primacy, or how they interact. (shrink)

Research on linguistic interaction suggests that two or more individuals can sometimes form adaptive and cohesive systems. We describe an “alignment system” as a loosely interconnected set of cognitive processes that facilitate social interactions. As a dynamic, multi-component system, it is responsive to higher-level cognitive states such as shared beliefs and intentions (those involving collective intentionality) but can also give rise to such shared cognitive states via bottom-up processes. As an example of putative group cognition we turn to transactive memory (...) and suggest how further research on alignment in these cases might reveal how such systems can be genuinely described as cognitive. Finally, we address a prominent critique of collective cognitive systems, arguing that there is much empirical and explanatory benefit to be gained from considering the possibility of group cognitive systems, especially in the context of small-group human interaction. (shrink)

Despite the challenges associated with virtually mediated communication, remote collaboration is a defining characteristic of online multiplayer gaming communities. Inspired by the teamwork exhibited by players in first-person shooter games, this study investigated the verbal and behavioral coordination of four-player teams playing a cooperative online video game. The game, Desert Herding, involved teams consisting of three ground players and one drone operator tasked to locate, corral, and contain evasive robot agents scattered across a large desert environment. Ground players could move (...) throughout the environment, while the drone operator’s role was akin to that of a “spectator” with a bird’s-eye view, with access to veridical information of the locations of teammates and the to-be-corralled agents. Categorical recurrence quantification analysis was used to measure the communication dynamics of teams as they completed the task. Demands on coordination were manipulated by varying the ground players’ ability to observe the environment with the use of game “fog.” Results show that catRQA was sensitive to changes to task visibility, with reductions in task visibility reorganizing how participants conversed during the game to maintain team situation awareness. The results are discussed in the context of future work that can address how team coordination can be augmented with the inclusion of artificial agents, as synthetic teammates. (shrink)

People coordinate body postures and gaze patterns during conversation. We review literature showing that (1) action embodies cognition, (2) postural coordination emerges spontaneously when two people converse, (3) gaze patterns influence postural coordination, (4) gaze coordination is a function of common ground knowledge and visual information that conversants believe they share, and (5) gaze coordination is causally related to mutual understanding. We then consider how coordination, generally, can be understood as temporarily coupled neuromuscular components that function as a collective unit (...) known as a coordinative structure in the motor control literature. We speculate that the coordination of gaze and body sway found in conversation may be understood as a cross‐person coordinative structure that embodies the goals of the joint action system. (shrink)

The present paper describes a joint action paradigm in which individuals or pairs utilized two computer keys to keep a dot stimulus moving inside a larger rectangle. Members of a pair could neither see nor hear each other. This paradigm allowed us to combine the discrete-trial type dependent variables commonly utilized by representational theorists, with the continuous, temporal dependence variables utilized by dynamical theorists. Analysis revealed that individuals kept the dot in the rectangle longer than dyads and did so by (...) moving it back and forth within the rectangle. Dyads, however, pressed their individual buttons as quickly as possible in order to keep the dot near the middle of the rectangle. These findings indicate that joint action constitutes a multi-scale phenomena that is best investigated via multiple, complementary methodologies versus single-measure, competing theories. (shrink)

For decades now a research question has firmly established itself as a staple of psychological and neuroscientific investigations on language, namely the question of whether and how bilingualism is cognitively beneficial, detrimental or neutral. As more and more studies appear every year, it seems as though the research question itself is firmly grounded and can be answered if only we use the right experimental manipulations and subject the data to the right analysis methods and interpretive lens. In this paper we (...) propose that, rather than merely improving prior methods in the pursuit of evidence in one direction or another, we would do well to carefully consider whether the research question itself is as firmly grounded as it might appear to be. We identify two bodies of research that suggest the research question to be highly problematic. In particular, drawing from work in sociolinguistics and in embodied cognitive science, we argue that the research question of whether bilingualism is cognitively advantageous or not is based on problematic assumptions about language and cognition. Once these assumptions are addressed head on, a straightforward answer to the question arises, but the question itself comes to seem to be a poor starting point for research. After examining why this is so, we conclude by exploring some implications for future research. (shrink)

In some areas of cognitive science we are confronted with ultrafast cognition, exquisite context sensitivity, and scale-free variation in measured cognitive activities. To move forward, we suggest a need to embrace this complexity, equipping cognitive science with tools and concepts used in the study of complex dynamical systems. The science of movement coordination has benefited already from this change, successfully circumventing analogous paradoxes by treating human activities as phenomena of self-organization. Therein, action and cognition are seen to be emergent in (...) ultrafast symmetry breaking across the brain and body; exquisitely constituted of the otherwise trivial details of history, context, and environment; and exhibiting the characteristic scale-free signature of self-organization. (shrink)

Conversation is supported by the beliefs that people have in common and the perceptual experience that they share. The visual context of a conversation has two aspects: the information that is available to each conversant, and their beliefs about what is present for each other. In our experiment, we separated these factors for the first time and examined their impact on a spontaneous conversation. We manipulated the fact that a visual scene was shared or not and the belief that a (...) visual scene was shared or not. Participants watched videos of actors talking about a controversial topic, then discussed their own views while looking at either a blank screen or the actors. Each believed (correctly or not) that their partner was either looking at a blank screen or the same images. We recorded conversants’ eye movements, quantified how they were coordinated, and analyzed their speech patterns. Gaze coordination has been shown to be causally related to the knowledge people share before a conversation, and the information they later recall. Here, we found that both the presence of the visual scene, and beliefs about its presence for another, influenced language use and gaze coordination. (shrink)

Important work on alignment systems has been applied to philosophical work on joint action by Tollefsen and Dale. This paper builds from and expands on their work. The first aim of the paper is to spell out how the empirical research on alignment may be integrated into philosophical theories of joint action. The second aim is then to develop a successful characterization of joint action, which spells out the difference between genuine joint action and simpler forms of coordination based on (...) alignment. I begin by introducing the empirical research and two definitions of joint action. I then argue that instead of using this research in conjunction with Searle’s account of collective intentionality, as Tollefsen and Dale suggest, we would be better served by applying this research to Gilbert’s account of plural subjects. In the final sections I distinguish between alignment, coordination, and joint action, clarify the roles of joint commitment and sub-personal alignment in joint action, and argue that... (shrink)

Dialog is a joint action at different levels. At the highest level, the goal of interlocutors is to align their mental representations. This emerges from joint activity at lower levels, both concerned with linguistic decisions (e.g., choice of words) and nonlinguistic processes (e.g., alignment of posture or speech rate). Because of the high‐level goal, the interlocutors are particularly concerned with close coupling at these lower levels. As we illustrate with examples, this means that imitation and entrainment are particularly pronounced during (...) interactive communication. We then argue that the mechanisms underlying such processes involve covert imitation of interlocutors’ communicative behavior, leading to emulation of their expected behavior. In other words, communication provides a very good example of predictive emulation, in a way that leads to successful joint activity. (shrink)

In the past several decades, psychologists and linguists have begun experimentally investigating linguistic pragmatic phenomena. They share the assumption that the best way to study the use of language in context incorporates an experimental methodology, here understood to comprise controlled studies and careful field observations. This article surveys some key projects in experimental pragmatics and relates these projects to ongoing philosophical discussions.

According to the hypotheses of distributed and extended cognition, remembering does not always occur entirely inside the brain but is often distributed across heterogeneous systems combining neural, bodily, social, and technological resources. These ideas have been intensely debated in philosophy, but the philosophical debate has often remained at some distance from relevant empirical research, while empirical memory research, in particular, has been somewhat slow to incorporate distributed/extended ideas. This situation, however, appears to be changing, as we witness an increasing level (...) of interaction between the philosophy and the empirical research. In this editorial, we provide a high-level historical overview of the development of the debates around the hypotheses of distributed and extended cognition, as well as relevant theory and empirical research on memory, considering both the role of memory in theoretical debates around distributed/extended ideas and strands of memory research that resonate with those ideas; we emphasize recent trends towards increased interaction, including new empirical paradigms for investigating distributed memory systems. We then provide an overview of the special issue itself, drawing out a number of general implications from the contributions, and conclude by sketching promising directions for future research on distributed memory. (shrink)

It has been claimed that a sense of us is presupposed for shared intentions to be possible. Searle introduced this notion together with the notion of the sense of the other. in joint action. It argues that the sense of the other is a necessary condition for a sense of us. Whereas thisarticle distinguishes between the “sense of the other” and the “sense of us” and elaborates on their role the sense of the other is immediate and automatic, the sense (...) of us can (but need not) arise between people and can (a) develop over time, (b) depend on the situation, and (c) involves several sufficient but not necessary processes. The article relies on research on core knowledge to better understand the sense of the other. It elaborates the sense of us using insights from cognitive science and social psychology. The article shows that the sense of the other and the sense of us can contribute to our understanding of the perception of possibilities for joint action and how individuals can come to experience actions and intentions as shared, even if the participants lack common knowledge. This leads to the conclusion that people are ordinarily socially oriented rather than individually. (shrink)

A variety of theoretical frameworks predict the resemblance of behaviors between two people engaged in communication, in the form of coordination, mimicry, or alignment. However, little is known about the time course of the behavior matching, even though there is evidence that dyads synchronize oscillatory motions (e.g., postural sway). This study examined the temporal structure of nonoscillatory actions—language, facial, and gestural behaviors—produced during a route communication task. The focus was the temporal relationship between matching behaviors in the interlocutors (e.g., facial (...) behavior in one interlocutor vs. the same facial behavior in the other interlocutor). Cross-recurrence analysis revealed that within each category tested (language, facial, gestural), interlocutors synchronized matching behaviors, at temporal lags short enough to provide imitation of one interlocutor by the other, from one conversational turn to the next. Both social and cognitive variables predicted the degree of temporal organization. These findings suggest that the temporal structure of matching behaviors provides low-level and low-cost resources for human interaction. (shrink)

Pickering & Garrod's (P&G's) framework assumes an efference copy based on the interlocutor's intentions. Yet, elaborate attribution of intentions may not always be necessary for online prediction. Instead, contextual cues such as speaker gaze can provide similar information with a lower demand on processing resources.

In online lectures, unlike in face-to-face lectures, teachers lack access to (nonverbal) cues to check if their students are still “with them” and comprehend the lecture. The increasing availability of low-cost eye-trackers provides a promising solution. These devices measure unobtrusively where students look and can visualize these data to teachers. These visualizations might inform teachers about students’ level of “with-me-ness” (i.e., do students look at the information that the teacher is currently talking about) and comprehension of the lecture, provided that (...) (1) gaze measures of “with-me-ness” are related to comprehension, (2) people not trained in eye-tracking can predict students’ comprehension from gaze visualizations, (3) we understand how different visualization techniques impact this prediction. We addressed these issues in two studies. In Study 1, 36 students watched a video lecture while being eye-tracked. The extent to which students looked at relevant information and the extent to which they looked at the same location as the teacher both correlated with students’ comprehension (score on an open question) of the lecture. In Study 2, 50 participants watched visualizations of students’ gaze (from Study 1), using six visualization techniques (dynamic and static versions of scanpaths, heatmaps, and focus maps) and were asked to predict students’ posttest performance and to rate their ease of prediction. We found that people can use gaze visualizations to predict learners’ comprehension above chance level, with minor differences between visualization techniques. Further research should investigate if teachers can act on the information provided by gaze visualizations and thereby improve students’ learning. (shrink)

Previous research showed that sensorimotor information affects the perception of properties associated with implied perceptual context during language comprehension. Three experiments addressed a novel question of whether perceptual context may contribute to a simulation of information about such out‐of‐sight objects as cast shadows. In Experiment 1, participants read a sentence that implied a particular shadow cast on a target (blinds vs. an open window) and then verified the picture of the object onto which a shadow was cast. Responses were faster (...) when the shadow of blinds cast on the object matched that implied by the sentence. However, the data did not show the same matching effect for pictures with cast shadows from an open window. In Experiments 2 and 3, we found that verification times for pictures with no cast shadows were faster when preceded by an “open window” sentence, thus suggesting that reading the sentence does not elicit a visual simulation of any specific shadow. Experiment 3 showed that the objects superimposed with a cast shadow of the blinds and blinds themselves were verified faster after reading a “blinds” sentence. However, the results of an order analysis showed the temporal stability of the “blinds shadows” effect, but the disappearance of the “blinds” effect in the second half of the data. We conclude that the results are compatible, to a lesser or greater extent, with multiple accounts, and discuss our findings in the context of a mental imagery view, a mental simulation view, and an amodal representation view. (shrink)

Although the neural systems that underlie spoken language are well-known, how they adapt to evolving social cues during natural conversations remains an unanswered question. In this work we investigate the neural correlates of face-to-face conversations between two individuals using functional near infrared spectroscopy and acoustical analyses of concurrent audio recordings. Nineteen pairs of healthy adults engaged in live discussions on two controversial topics where their opinions were either in agreement or disagreement. Participants were matched according to their a priori opinions (...) on these topics as assessed by questionnaire. Acoustic measures of the recorded speech including the fundamental frequency range, median fundamental frequency, syllable rate, and acoustic energy were elevated during disagreement relative to agreement. Consistent with both the a priori opinion ratings and the acoustic findings, neural activity associated with long-range functional networks, rather than the canonical language areas, was also differentiated by the two conditions. Specifically, the frontoparietal system including bilateral dorsolateral prefrontal cortex, left supramarginal gyrus, angular gyrus, and superior temporal gyrus showed increased activity while talking during disagreement. In contrast, talking during agreement was characterized by increased activity in a social and attention network including right supramarginal gyrus, bilateral frontal eye-fields, and left frontopolar regions. Further, these social and visual attention networks were more synchronous across brains during agreement than disagreement. Rather than localized modulation of the canonical language system, these findings are most consistent with a model of distributed and adaptive language-related processes including cross-brain neural coupling that serves dynamic verbal exchanges. (shrink)

Integrating different perspectives is a sophisticated strategy for developing constructive interactions in collaborative problem solving. However, cognitive aspects such as individuals’ knowledge and bias often obscure group consensus and produce conflict. This study investigated collaborative problem solving, focusing on a group member interacting with another member having a different perspective. It was predicted that mavericks might mitigate disadvantages and facilitate perspective taking during problem solving. Thus, 344 university students participated in two laboratory-based experiments by engaging in a simple rule-discovery task (...) that raised conflicts among perspectives. They interacted with virtual partners whose conversations were controlled by multiple conversational agents. Results show that when participants interacted with a maverick during the task, they were able to take others’ perspectives and integrate different perspectives to solve the problem. Moreover, when participants interacted in groups with a positive mood, groups with a maverick outperformed groups having several perspectives. (shrink)

Our everyday interactions increasingly involve both embodied face-to-face communication and various forms of mediated and distributed communication such as email, skype, and facebook. In daily face-to-face communications, we are connected in rhythm and synchrony at multiple levels ranging from the moment-by-moment continuity of timed syllables to emergent body and vocal rhythms of pragmatic sense-making. Our human capacity to synchronize with each other may be essential for our survival as social beings. Moving our bodies and voices together in time embodies a (...) potent pragmatic purpose that of being together. In this synchrony of self with other, witnessing and being present become part of each other. There is growing research into how rhythm and synchrony operate in embodied face-to-face interaction and this provides parameters for investigating the relations and differences in how we connect and are socially present in the embodied and distributed settings, and understanding the effect of one setting upon the other. This paper explores the arena of research into rhythm in human interaction, musical and linguistic, with a focus on the movements of body and voice. It draws together salient issues and ideas that would form the basis for a framework of rhythm in embodied interaction. (shrink)

Dialog is a joint action at different levels. At the highest level, the goal of interlocutors is to align their mental representations. This emerges from joint activity at lower levels, both concerned with linguistic decisions (e.g., choice of words) and nonlinguistic processes (e.g., alignment of posture or speech rate). Because of the high‐level goal, the interlocutors are particularly concerned with close coupling at these lower levels. As we illustrate with examples, this means that imitation and entrainment are particularly pronounced during (...) interactive communication. We then argue that the mechanisms underlying such processes involve covert imitation of interlocutors’ communicative behavior, leading to emulation of their expected behavior. In other words, communication provides a very good example of predictive emulation, in a way that leads to successful joint activity. (shrink)

Beyond the observation that both speakers and listeners rapidly inspect the visual targets of referring expressions, it has been argued that such gaze may constitute part of the communicative signal. In this study, we investigate whether a speaker may, in principle, exploit listener gaze to improve communicative success. In the context of a virtual environment where listeners follow computer-generated instructions, we provide two kinds of support for this claim. First, we show that listener gaze provides a reliable real-time index of (...) understanding even in dynamic and complex environments, and on a per-utterance basis. Second, we show that a language generation system that uses listener gaze to provide rapid feedback improves overall task performance in comparison with two systems that do not use gaze. Aside from demonstrating the utility of listener gaze in situated communication, our findings open the door to new methods for developing and evaluating multi-modal models of situated interaction. (shrink)

Journal Name: Cognitive Linguistics Issue: Ahead of print.

Cognitive Science, Volume 45, Issue 11, November 2021.

Numerous philosophical theories of joint agency and its intentional structure have been developed in the past few decades. These theories have offered accounts of joint agency that appeal to higher-level states that are?shared? in some way. These accounts have enhanced our understanding of joint agency, yet there are a number of lower-level cognitive phenomena involved in joint action that philosophers rarely acknowledge. In particular, empirical research in cognitive science has revealed that when individuals engage in a joint activity such as (...) conversation or joint problem solving, they become aligned at multiple levels. We argue that this phenomenon of alignment is crucial to understanding joint actions and should be integrated with philosophical approaches. In this paper, we sketch a possible integration, and draw out its implications for understanding of joint agency and collective intentionality. The result is a process-based, dynamic account of joint action that integrates both low-level and high-level states, and seeks to capture the separate processes of how a joint action is initiated and sustained. (shrink)

Task-unrelated thought (TUT) occurs frequently in our daily lives and across a range of tasks, but we know little about how this phenomenon arises during and influences the way we communicate. Conversations also provide a novel opportunity to assess the alignment (or divergence) in TUT during dyadic interactions. We conducted a study to determine: (a) the frequency of TUT during conversation as well as how partners align/diverge in their rates of TUT, (b) the subjective and behavioral correlates of TUT and (...) TUT divergence during conversation, and (c) if perceived social group identity impacts TUT and TUT divergence during conversation. We used a minimal groups induction procedure to assign participants (N = 126) to either an ingroup, outgroup, or control condition. We then asked them to converse with one another via a computer-mediated text chat application for 10 min while self-reporting TUTs. On average, participants reported TUT about once every 2 min; however, this rate was lower for participants in the ingroup condition, compared to the control condition. Conversational pairs in the ingroup condition were also aligned more in their rates of TUT compared to the outgroup condition. Finally, we discuss subjective and behavioral correlates of TUT and TUT divergence in conversations, such as valence, turn-taking ratios, and topic shifts. (shrink)

When people communicate, they coordinate a wide range of linguistic and non-linguistic behaviors. This process of coordination is called alignment, and it is assumed to be fundamental to successful communication. In this paper, we question this assumption and investigate whether disalignment is a more successful strategy in some cases. More specifically, we hypothesize that alignment correlates with task success only when communication is interactive. We present results from a spot-the-difference task in which dyads of interlocutors have to decide whether they (...) are viewing the same scene or not. Interactivity was manipulated in three conditions by increasing the amount of information shared between interlocutors. We use recurrence quantification analysis to measure the alignment between the scan-patterns of the interlocutors. We found that interlocutors who could not exchange feedback aligned their gaze more, and that increased gaze alignment correlated with decreased task success in this case. When feedback was possible, in contrast, interlocutors utilized it to better organize their joint search strategy by diversifying visual attention. This is evidenced by reduced overall alignment in the minimal feedback and full dialogue conditions. However, only the dyads engaged in a full dialogue increased their gaze alignment over time to achieve successful performances. These results suggest that alignment per se does not imply communicative success, as most models of dialogue assume. Rather, the effect of alignment depends on the type of alignment, on the goals of the task, and on the presence of feedback. (shrink)

During communication, we form assumptions about what our communication partners know and believe. Information that is mutually known between the discourse partners—their common ground—serves as a backdrop for successful communication. Here we present an introduction to the focus of this topic, which is the role of memory in common ground and language use. Two types of questions emerge as central to understanding the relationship between memory and common ground, specifically questions having to do with the representation of common ground in (...) memory, and the use of common ground during language processing. (shrink)

We hypothesize that effective collaboration is facilitated when individuals and environmental components form a synergy where they work together and regulate one another to produce stable patterns of behavior, or regularity, as well as adaptively reorganize to form new behaviors, or irregularity. We tested this hypothesis in a study with 32 triads who collaboratively solved a challenging visual computer programming task for 20 min following an introductory warm‐up phase. Multidimensional recurrence quantification analysis was used to examine fine‐grained (i.e., every 10 (...) s) collective patterns of regularity across team members' speech rate, body movement, and team interaction with the shared user interface. We found that teams exhibited significant patterns of regularity as compared to shuffled baselines, but there were no systematic trends in regularity across time. We also found that periods of regularity were associated with a reduction in overall behavior. Notably, the production of irregular behavior predicted expert‐coded metrics of collaborative activity, such as teams' ability to construct shared knowledge and effectively negotiate and coordinate execution of solutions, net of overall behavioral production and behavioral self‐similarity. Our findings support the theory that groups can interact to form interpersonal synergies and indicate that information about system‐level dynamics is a viable way to understand and predict effective collaborative processes. (shrink)