The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have (...) limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the nonlinear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique’s application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a “Programming Basics” module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data. (shrink)

Multiple theories of problem-solving hypothesize that there are distinct qualitative phases exhibited during effective problem-solving. However, limited research has attempted to identify when transitions between phases occur. We integrate theory on collaborative problem-solving with dynamical systems theory suggesting that when a system is undergoing a phase transition it should exhibit a peak in entropy and that entropy levels should also relate to team performance. Communications from 40 teams that collaborated on a complex problem were coded for occurrence of problem-solving processes. (...) We applied a sliding window entropy technique to each team's communications and specified criteria for identifying data points that qualify as peaks and determining which peaks were robust. We used multilevel modeling, and provide a qualitative example, to evaluate whether phases exhibit distinct distributions of communication processes. We also tested whether there was a relationship between entropy values at transition points and CPS performance. We found that a proportion of entropy peaks was robust and that the relative occurrence of communication codes varied significantly across phases. Peaks in entropy thus corresponded to qualitative shifts in teams’ CPS communications, providing empirical evidence that teams exhibit phase transitions during CPS. Also, lower average levels of entropy at the phase transition points predicted better CPS performance. We specify future directions to improve understanding of phase transitions during CPS, and collaborative cognition, more broadly. (shrink)

Complex work in teams requires coordination across team members and their technology as well as the ability to change and adapt over time to achieve effective performance. To support such complex interactions, recent efforts have worked toward the design of adaptive human-autonomy teaming systems that can provide feedback in or near real time to achieve the desired individual or team results. However, while significant advancements have been made to better model and understand the dynamics of team interaction and its relationship (...) with task performance, appropriate measures of team coordination and computational methods to detect changes in coordination have not yet been widely investigated. Having the capacity to measure coordination in real time is quite promising as it provides the opportunity to provide adaptive feedback that may influence and regulate teams’ coordination patterns and, ultimately, drive effective team performance. A critical requirement to reach this potential is having the theoretical and empirical foundation from which to do so. Therefore, the first goal of the paper is to review approaches to coordination dynamics, identify current research gaps, and draw insights from other areas, such as social interaction, relationship science, and psychotherapy. The second goal is to collate extant work on feedback and advance ideas for adaptive feedback systems that have potential to influence coordination in a way that can enhance the effectiveness of team interactions. In addressing these two goals, this work lays the foundation as well as plans for the future of human-autonomy teams that augment team interactions using coordination-based measures. (shrink)

In the performing arts, such as music and dance performances, people actively interact with each other and show their exciting performances. Some studies have proposed that this interaction is a social origin of the performing arts. Some have further investigated this phenomenon based on the synchronization and coordination theory. Though the majority of these studies have focused on the collaborative context, several genres of the performing arts, such as jazz sessions and breakdance battles, have a competitive context. Several studies have (...) suggested that, in this competitive context, performers actively interact with each other and construct some correspondence. Moreover, a few recent studies focusing on competitive conversations, such as debates, have shown that, compared to people's interactions in collaborative conversations, people in competitive contexts frequently coordinate their behaviors in complicated ways. However, the interaction and coordination among performers in these competitive contexts have not been sufficiently investigated. Therefore, we investigated the coordination of expert breakdancers in battle scenes and measured their rhythmic movements using a motion capture system. We calculated the relative phase of the rhythmic movements between two dancers to investigate their coordination. The results showed that the dancers' rhythmic movements tended to synchronize in an anti-phase fashion, which means that there were similarities as well as differences between the two dancers' rhythmic movements. Furthermore, this pattern of coordination changed dynamically as time elapsed, from an in-phase synchronization or leader-follower relationships to an anti-phase synchronization and then leader-follower relationships. (shrink)

We observe that approaches to intersubjectivity, involving mirror neurons and involving emulation and prediction, have eclipsed discussion of those same mechanisms for achieving coordination between the two hemispheres of the human brain. We explore some of the implications of the suggestion that the mutual modelling of the two situated hemispheres is a productive place to start in understanding the phylogenetic and ontogenetic development of cognition and of intersubjectivity.

For decades now a research question has firmly established itself as a staple of psychological and neuroscientific investigations on language, namely the question of whether and how bilingualism is cognitively beneficial, detrimental or neutral. As more and more studies appear every year, it seems as though the research question itself is firmly grounded and can be answered if only we use the right experimental manipulations and subject the data to the right analysis methods and interpretive lens. In this paper we (...) propose that, rather than merely improving prior methods in the pursuit of evidence in one direction or another, we would do well to carefully consider whether the research question itself is as firmly grounded as it might appear to be. We identify two bodies of research that suggest the research question to be highly problematic. In particular, drawing from work in sociolinguistics and in embodied cognitive science, we argue that the research question of whether bilingualism is cognitively advantageous or not is based on problematic assumptions about language and cognition. Once these assumptions are addressed head on, a straightforward answer to the question arises, but the question itself comes to seem to be a poor starting point for research. After examining why this is so, we conclude by exploring some implications for future research. (shrink)

When people are engaged in social interaction, they can repeat aspects of each other’s communicative behavior, such as words or gestures. This kind of behavioral alignment has been studied across a wide range of disciplines and has been accounted for by diverging theories. In this paper, we review various operationalizations of lexical and gestural alignment. We reveal that scholars have fundamentally different takes on when and how behavior is considered to be aligned, which makes it difficult to compare findings and (...) draw uniform conclusions. Furthermore, we show that scholars tend to focus on one particular dimension of alignment (traditionally, whether two instances of behavior overlap in form), while other dimensions remain understudied. This hampers theory testing and building, which requires a well‐defined account of the factors that are central to or might enhance alignment. To capture the complex nature of alignment, we identify five key dimensions to formalize the relationship between any pair of behavior: time, sequence, meaning, form, and modality. We show how assumptions regarding the underlying mechanism of alignment (placed along the continuum of priming vs. grounding) pattern together with operationalizations in terms of the five dimensions. This integrative framework can help researchers in the field of alignment and related phenomena (including behavior matching, mimicry, entrainment, and accommodation) to formulate their hypotheses and operationalizations in a more transparent and systematic manner. The framework also enables us to discover unexplored research avenues and derive new hypotheses regarding alignment. (shrink)

Cognitive Science, Volume 46, Issue 10, October 2022.

The literature on coordination within and between individuals is reviewed, with an emphasis on the inherent transience of coordination patterns in behavioral activity. This transience is integral to understanding cognitive activity as flexible patterns of coordination in brain, body, and environment. Kello reviews the articles in this special issue as contributions to understanding the role of context in shaping or interpreting coordination patterns in human behavior.

We address what it means to “engage in a relationship” and suggest Social Breathing as a model of immersing ourselves in the metaphorical social air around us, which is necessary for shared intention and joint action. We emphasize how emergent properties of social systems arise, such as the shared culture of groups, which cannot be reduced to the individuals involved. We argue that the processes involved in Social Breathing are: automatic, implicit, temporal, in the form of mutual bi-directional interwoven exchanges (...) between social partners and embodied in the coordination of the brains and behaviors of social partners. We summarize cross-disciplinary evidence suggesting that these processes involve a multi-person whole-brain-body network which is critical for the development of both we-ness and relational skills. We propose that Social Breathing depends on each individual’s ability to sustain multimodal interwovenness, thus providing a theoretical link between social neuroscience and relational/multi-person psychology. We discuss how the model could guide research on autism, relationships, and psychotherapy. (shrink)

This study investigates interpersonal processes underlying dialog by comparing two approaches, interactive alignment and interpersonal synergy, and assesses how they predict collective performance in a joint task. While the interactive alignment approach highlights imitative patterns between interlocutors, the synergy approach points to structural organization at the level of the interaction—such as complementary patterns straddling speech turns and interlocutors. We develop a general, quantitative method to assess lexical, prosodic, and speech/pause patterns related to the two approaches and their impact on collective (...) performance in a corpus of task-oriented conversations. The results show statistical presence of patterns relevant for both approaches. However, synergetic aspects of dialog provide the best statistical predictors of collective performance and adding aspects of the alignment approach does not improve the model. This suggests that structural organization at the level of the interaction plays a crucial role in task-oriented conversations, possibly constraining and integrating processes related to alignment. (shrink)

The literature on social interactions has shown that participants coordinate not only at the behavioral but also at the physiological and neural levels, and that this coordination gives a temporal structure to the individual and social dynamics. However, it has not been fully explored whether such temporal patterns emerge during interpersonal coordination beyond dyads, whether this phenomenon arises from complex cognitive mechanisms or from relatively simple rules of behavior, or which are the sociocultural processes that underlie this phenomenon. We review (...) the evidence for the existence of group-level rhythmic patterns that result from social interactions and argue that the complexity of group dynamics can lead to temporal regularities that cannot be predicted from the individual periodicities: an emergent collective rhythm. Moreover, we use this interpretation of the literature to discuss how taking into account the sociocultural niche in which individuals develop can help explain the seemingly divergent results that have been reported on the social influences and consequences of interpersonal coordination. We make recommendations on further research to test these arguments and their relationship to the feeling of belonging and assimilation experienced during group dynamics. (shrink)

Gestures and speech are clearly synchronized in many ways. However, previous studies have shown that the semantic similarity between gestures and speech breaks down as people approach transitions in understanding. Explanations for these gesture–speech mismatches, which focus on gestures and speech expressing different cognitive strategies, have been criticized for disregarding gestures’ and speech's integration and synchronization. In the current study, we applied three different perspectives to investigate gesture–speech synchronization in an easy and a difficult task: temporal alignment, semantic similarity, and (...) complexity matching. Participants engaged in a simple cognitive task and were assigned to either an easy or a difficult condition. We automatically measured pointing gestures, and we coded participant's speech, to determine the temporal alignment and semantic similarity between gestures and speech. Multifractal detrended fluctuation analysis was used to determine the extent of complexity matching between gestures and speech. We found that task difficulty indeed influenced gesture–speech synchronization in all three domains. We thereby extended the phenomenon of gesture–speech mismatches to difficult tasks in general. Furthermore, we investigated how temporal alignment, semantic similarity, and complexity matching were related in each condition, and how they predicted participants’ task performance. Our study illustrates how combining multiple perspectives, originating from different research areas (i.e., coordination dynamics, complexity science, cognitive psychology), provides novel understanding about cognitive concepts in general and about gesture–speech synchronization and task difficulty in particular. (shrink)

Through theoretical discussion, literature review, and a computational model, this paper poses a challenge to the notion that perspective-taking involves a fixed architecture in which particular processes have priority. For example, considerable work has shown that egocentric perspectives can arise more quickly, with other perspectives (such as of task partners) emerging only secondarily. This theoretical dichotomy is challenged here, and we propose a general view of perspective-taking as an emergent phenomenon governed by the interplay among several cognitive mechanisms. We first (...) describe the pervasive relevance of perspective-taking to cognitive science. A dynamical systems model is then introduced which explicitly formulates the timescale interaction proposed. Implications are discussed, with ideas for future empirical research. (shrink)

Task-unrelated thought (TUT) occurs frequently in our daily lives and across a range of tasks, but we know little about how this phenomenon arises during and influences the way we communicate. Conversations also provide a novel opportunity to assess the alignment (or divergence) in TUT during dyadic interactions. We conducted a study to determine: (a) the frequency of TUT during conversation as well as how partners align/diverge in their rates of TUT, (b) the subjective and behavioral correlates of TUT and (...) TUT divergence during conversation, and (c) if perceived social group identity impacts TUT and TUT divergence during conversation. We used a minimal groups induction procedure to assign participants (N = 126) to either an ingroup, outgroup, or control condition. We then asked them to converse with one another via a computer-mediated text chat application for 10 min while self-reporting TUTs. On average, participants reported TUT about once every 2 min; however, this rate was lower for participants in the ingroup condition, compared to the control condition. Conversational pairs in the ingroup condition were also aligned more in their rates of TUT compared to the outgroup condition. Finally, we discuss subjective and behavioral correlates of TUT and TUT divergence in conversations, such as valence, turn-taking ratios, and topic shifts. (shrink)

Coordinated group behaviour can result in conflict or social cohesion. Thus having a better understanding of coordination in social groups could help us tackle some of our most challenging social problems. Historically, the most common way to study group behaviour is to break it down into sub-processes, such as cognition and emotion, then ideally manipulate them in a social context in order to predict some behaviour such as liking versus distrusting a target person. This approach has gotten us partway to (...) understanding many important collective behaviours, but I argue that making major changes in the world will require a more integrated approach. In this review, I introduce dynamic systems theory, with a focus on interpersonal systems, where all the processes we typically study in individuals, such as cognition and emotion, become intertwined between social partners over time. I focus on the concept of coordination, defined as a temporal correlation between interacting components of a system (or systems) arising due to coupling between them. Finally, I show how this perspective could be used to guide investigations of social problems such as polarisation. (shrink)

The sensorimotor approach to perception addresses various aspects of perceptual experience, but not the subjectivity of intentional action. Conversely, the problem that current accounts of the sense of agency deal with is primarily one of subjectivity. But the proposed models, based on internal signal comparisons, arguably fail to make the transition from subpersonal computations to personal experience. In this paper we suggest an alternative direction towards explaining the sense of agency by braiding three theoretical strands: a world-involving, dynamical interpretation of (...) the sensorimotor approach, an enactive description of sensorimotor agency as contrasted with organic agency in general, and a dynamical theory of equilibration within and between sensorimotor schemes. On this new account, the sense of oneself as the author of one’s own actions corresponds to what we experience during the ongoing adventure of establishing, losing, and re-establishing meaningful interactions with the world. The meaningful relation between agent and world is given by the precarious constitution of sensorimotor agency as a self-asserting network of schemes and dispositions. Acts are owned as they adaptively assert the constitution of the agent. Thus, awareness for different aspects of agency experience, such as the initiation of action, the effort exerted in controlling it, or the achievement of the desired effect, can be accounted for by processes involved in maintaining the sensorimotor organization that enables these interactions with the world. We discuss these processes in detail from a non-representational, dynamical perspective and show how they cohere with the personal experience of agency. (shrink)

In this article, moving from being to becoming, we construe the ‘self’ as a dynamic process rather than as a static entity. To this end we draw on dialectics and Bayesian accounts of cognition. The former allows us to holistically consider the ‘self’ as the interplay between internalization and externalization and the latter to operationalize our suggestion formally. Internalization is considered here as the co-construction of bodily hierarchical models of the world and the organism, while externalization is taken as the (...) collective transformation of the world. We do not consider these processes as sequentially linked, but rather as a dialectic between the collective and the individual. This leads us to the suggestion of the self as a historical product of dialectical attunement across multiple time scales, from species evolution and culture to individual development and everyday learning. Subsequently, we describe concrete means for empirically testing our proposal in the form of two-person psychophysiology and multi-level analyses of intersubjectivity. Taken together, we suggest that a fine-grained analysis of social interaction might allow us to reconsider the ‘self’ beyond the static individual, i.e. how it emerges and manifests itself in social relations. Such an approach, we believe, could be relevant in multiple fields, from ethics and psychiatry to pedagogy and artificial intelligence. (shrink)

In this article, moving from being to becoming, we construe the ‘self’ as a dynamic process rather than as a static entity. To this end we draw on dialectics and Bayesian accounts of cognition. The former allows us to holistically consider the ‘self’ as the interplay between internalization and externalization and the latter to operationalize our suggestion formally. Internalization is considered here as the co-construction of bodily hierarchical models of the world and the organism, while externalization is taken as the (...) collective transformation of the world. We do not consider these processes as sequentially linked, but rather as a dialectic between the collective and the individual. This leads us to the suggestion of the self as a historical product of dialectical attunement across multiple time scales, from species evolution and culture to individual development and everyday learning. Subsequently, we describe concrete means for empirically testing our proposal in the form of two-person psychophysiology and multi-level analyses of intersubjectivity. Taken together, we suggest that a fine-grained analysis of social interaction might allow us to reconsider the ‘self’ beyond the static individual, i.e. how it emerges and manifests itself in social relations. Such an approach, we believe, could be relevant in multiple fields, from ethics and psychiatry to pedagogy and artificial intelligence. (shrink)

Using fractal analyses to study events allows us to capture the scale-independence of those events, that is, no matter at which level we study a phenomenon, we should get roughly the same results because events exhibit similar structure across scales. This is demonstrably true in mathematical fractals but is less assured in behavioral fractals. The current research directly tests the scale-independence hypothesis in the behavioral domain by exploring the fractal structure of aggression, a social phenomenon comprising events that span temporal (...) scales from minutes of face-to-face arguments to centuries of international armed conflicts. Using publicly available data, we examined the temporal fractal structure of four scales of aggression: wars, riots, violent crimes, and body movement during arguments. Our results lend mixed support to the scale-independence hypothesis and provide insight into the self-organization of human interactions. (shrink)

Long‐range correlation is a general class of coordination pattern found to be common to the intrinsic dynamics of complex systems, including human behavior. Balasubramaniam, Hove, and Médé investigate intrinsic dynamics in repeated finger movements, and they find that different measures of movement dynamics yield different long‐range correlations. Results shed light on the way that coordination patterns are expressed as a function of measurement context.

Recent studies of naturalistic face‐to‐face communication have demonstrated coordination patterns such as the temporal matching of verbal and non‐verbal behavior, which provides evidence for the proposal that verbal and non‐verbal communicative control derives from one system. In this study, we argue that the observed relationship between verbal and non‐verbal behaviors depends on the level of analysis. In a reanalysis of a corpus of naturalistic multimodal communication (Louwerse, Dale, Bard, & Jeuniaux, ), we focus on measuring the temporal patterns of specific (...) communicative behaviors in terms of their burstiness. We examined burstiness estimates across different roles of the speaker and different communicative modalities. We observed more burstiness for verbal versus non‐verbal channels, and for more versus less informative language subchannels. Using this new method for analyzing temporal patterns in communicative behaviors, we show that there is a complex relationship between verbal and non‐verbal channels. We propose a “temporal heterogeneity” hypothesis to explain how the language system adapts to the demands of dialog. (shrink)