There is plenty of evidence that speech and gesture form a tightly integrated system, as reflected in parallelisms in language production, comprehension, and development ( McNeill, 1992 ; Kendon, 2004 ). Yet, it is a common assumption that speakers use gestures to compensate for their expressive difficulties, a notion found in developmental studies of both first and second language acquisition, and in theoretical proposals concerning the gesture-speech relationship. If gestures are compensatory, they should mainly occur in disfluent stretches of speech. (...) However, the evidence is sparse and conflicting. This study extends previous studies and tests the putative compensatory role of gestures by comparing the gestural behavior in fluent vs. disfluent stretches of narratives by competent speakers in two languages (Dutch and Italian), and by language learners (children and adult L2 learners). The results reveal that (1) in all groups speakers overwhelmingly produce gestures during fluent speech and only rarely during disfluencies. However, L2 learners are significantly more likely to gesture in disfluency than the other groups; (2) in all groups gestures during disfluencies tend to be holds; (3) in all groups the rare gestures completed in disfluencies have both referential and pragmatic functions. Overall, the data strongly suggest that when speech stops, so does gesture. The findings constitute an important challenge to both gesture and language acquisition theories assuming a mainly (lexical) compensatory role for (referential) gestures. Instead, the results provide strong support for the notion that speech and gestures form an integrated system. (shrink)

This study investigates whether addressees visually attend to speakers’ gestures in interaction and whether attention is modulated by changes in social setting and display size. We compare a live face-to-face setting to two video conditions. In all conditions, the face dominates as a fixation target and only a minority of gestures draw fixations. The social and size parameters affect gaze mainly when combined and in the opposite direction from the predicted with fewer gestures fixated on video than live. Gestural holds (...) and speakers’ gaze at their own gestures reliably attract addressees’ fixations in all conditions. The attraction force of holds is unaffected by changes in social and size parameters, suggesting a bottom-up response, whereas speaker-fixated gestures draw significantly less attention in both video conditions, suggesting a social effect for overt gaze-following and visual joint attention. The study provides and validates a video-based paradigm enabling further experimental but ecologically valid explorations of cross-modal information processing. (shrink)

A key challenge when learning language in naturalistic circumstances is to extract linguistic information from a continuous stream of speech. This study investigates the predictors of such implicit learning among adults exposed to a new language in a new modality. Sign-naïve participants were shown a 4-min weather forecast in Swedish Sign Language. Subsequently, we tested their ability to recognise 22 target sign forms that had been viewed in the forecast, amongst 44 distractor signs that had not been viewed. The target (...) items differed in their occurrence frequency in the forecast and in their degree of iconicity. The results revealed that both frequency and iconicity facilitated recognition of target signs cumulatively. The adult mechanism for language learning thus operates similarly on sign and spoken languages as regards frequency, but also exploits modality-salient properties, for example iconicity for sign languages. Individual differences in cognitive skills and language learning background did not predict recognition. The properties of the input thus influenced adults’ language learning abilities at first exposure more than individual differences. (shrink)

As humans interact in the world, they often orient one another's attention to objects through the use of spoken demonstrative expressions and head and/or hand movements to point to the objects. Although indicating behaviors have frequently been studied in lab settings, we know surprisingly little about how demonstratives and pointing are used to coordinate attention in large-scale space and in natural contexts. This study investigates how speakers of Quiahije Chatino, an indigenous language of Mexico, use demonstratives and pointing to give (...) directions to named places in large-scale space across multiple scales. The results show that the use and coordination of demonstratives and pointing change as the scale of search space for the target grows. At larger scales, demonstratives and pointing are more likely to occur together, and the two signals appear to manage different aspects of the search for the target: demonstratives orient attention primarily to the gesturing body, while pointing provides cues for narrowing the search space. These findings underscore the distinct contributions of speech and gesture to the linguistic composite, while illustrating the dynamic nature of their interplay.s in Spanish and Quiahije Chatino are provided as appendices.Se incluyen como apéndices resúmenes en español y en el chatino de San Juan Quiahije. SonGktyiCreCinH, ngyaqCskaEktyiCnoEndaHsonBnaFngaJnoIngyaqCloEktyiCreC, ngyaqCranFchaqExlyaKqoEchaqFjnyaJnoAndywiqArenqAKchinAKyqyaC. (shrink)

Studies of native syntactic processing often target phrase structure violations that do not occur in natural production. In contrast, this study examines how variation in basic word order is processed, looking specifically at structures traditionally labelled as violations but that do occur naturally. We examined Swedish verb-second and verb-third word order processing in adult native Swedish speakers, manipulating sentence-initial adverbials in acceptability judgements, in simultaneously recorded event-related potentials to visually presented sentences and in a written sentence completion task. An initial (...) corpus study showed that the adverbials differ in frequency in fronted position, and although all occur mainly with V2 word order, kanske occurs more frequently with V3 in natural production than both idag and hemma. The experimental results reflected these patterns such that V2 sentences were overall more frequently produced and were deemed more acceptable than V3 sentences. The ERP results consisted of a biphasic N400/P600 response to V3 word order that indicated effects of word retrieval and sentence reanalysis. We also found consistent effects of adverbials. As predicted, V3 was produced more frequently and judged as more acceptable in Kanske sentences than in sentences with the other two adverbials. The ERP analyses showed stronger effects for idag and hemma with V3, especially regarding the P600. The results suggest that the naturally occurring word order ‘violation’, V3 with kanske, is processed differently than V3 with other adverbials where the V2 norm is stronger. Moreover, these patterns are related to individuals’ own production patterns. Overall, the results suggest a more varied native word order processing than previously reported. (shrink)

Despite a wealth of studies on effects of switch locations in code-switching (CS), we know relatively little about how structural factors such as switch location and extralinguistic factors such as directionality preferences may jointly modulate CS (cf., Stell and Yapko,2015). Previous findings in the nominal domain suggest that within-constituent switching (within the noun phrase) may be easier to process than between-constituent switching (a structural effect), and that there may also be directionality effects with switches preferred only in one language direction (...) (an extra-linguistic effect). In this study we examine a different domain, namely how VP-external (preverbal) vs. VP-internal (postverbal) switch location and switch directionality affects the processing of Papiamentu–Dutch mixed subject-verb-object (SVO) sentences. We manipulated switch location (preverbal/postverbal), and directionality of switch (PD/DP) and tested 50 Papiamentu–Dutch bilinguals on an auditory sentence matching task. The results from the mixed conditions showed no effect of switch location. Instead, we found only an effect of directionality and in an unexpected direction for this population, with switches from Dutch to Papiamentu being processed faster than switches from Papiamentu to Dutch regardless of switch location. The results highlight the importance of taking extralinguistic factors into account, but also the challenges of studying CS, particularly in lesser studied speech communities, and the need for a data-driven, cross-disciplinary approach to the study of CS. (shrink)