Psycholinguistic studies often look at the production of referring expressions in interactive settings, but so far few referring expression generation algorithms have been developed that are sensitive to earlier references in an interaction. Rather, such algorithms tend to rely on domain-dependent preferences for both content selection and linguistic realization. We present three experiments showing that humans may opt for dispreferred attributes and dispreferred modifier orderings when these were primed in a preceding interaction (without speakers being consciously aware of this). In (...) addition, we show that speakers are more likely to produce overspecified references, including dispreferred attributes (although minimal descriptions with preferred attributes would suffice), when these were similarly primed. (shrink)

This study investigates to what extent the amount of variation in a visual scene causes speakers to mention the attribute color in their definite target descriptions, focusing on scenes in which this attribute is not needed for identification of the target. The results of our three experiments show that speakers are more likely to redundantly include a color attribute when the scene variation is high as compared with when this variation is low (even if this leads to overspecified descriptions). We (...) argue that these findings are problematic for existing algorithms that aim to automatically generate psychologically realistic target descriptions, such as the Incremental Algorithm, as these algorithms make use of a fixed preference order per domain and do not take visual scene variation into account. (shrink)

We investigate whether smile mimicry and emotional contagion are evident in non-text-based computer-mediated communication (CMC). Via an ostensibly real-time audio-visual CMC platform, participants interacted with a confederate who either smiled radiantly or displayed a neutral expression throughout the interaction. Automatic analyses of expressions displayed by participants indicated that smile mimicry was at play: A higher level of activation of the facial muscle that characterises genuine smiles was observed among participants who interacted with the smiling confederate than among participants who interacted (...) with the unexpressive confederate. However, there was no difference in the self-reported level of joviality between participants in the two conditions. Our findings demonstrate that people mimic smiles in audio-visual CMC, but that even though the diffusion of emotions has been documented in text-based CMC in previous studies, we find no convincing support for the phenomenon of emotional contagion in non-text-based CMC. (shrink)

Past research has sought to elucidate how speakers and addressees establish common ground in conversation, yet few studies have focused on how visual cues such as co-speech gestures contribute to this process. Likewise, the effect of cognitive constraints on multimodal grounding remains to be established. This study addresses the relationship between the verbal and gestural modalities during grounding in referential communication. We report data from a collaborative task where repeated references were elicited, and a time constraint was imposed to increase (...) cognitive load. Our results reveal no differential effects of repetition or cognitive load on the semantic-based gesture rate, suggesting that representational gestures and speech are closely coordinated during grounding. However, gestures and speech differed in their execution, especially under time pressure. We argue that speech and gesture are two complementary streams that might be planned in conjunction but that unfold independently in later stages of language production, with speakers emphasizing the form of their gestures, but not of their words, to better meet the goals of the collaborative task. (shrink)

It has often been observed that color is a highly preferred attribute for use in distinguishing descriptions, that is, referring expressions produced with the purpose of identifying an object within a visual scene. However, most of these observations were based on visual displays containing only colors that were maximally different in hue and for which the language of experimentation possessed basic color terms. The experiments described in this paper investigate whether speakers’ preference for color is reduced if the color of (...) the target referent is similar to that of the distractors. Because colors that look similar are often also harder to distinguish linguistically, we also examine the impact of the codability of color values. As a third factor, we investigate the salience of available alternative attributes and its impact on the use of color. The results of our experiments show that, while speakers are indeed less likely to use color when the colors in a display are similar, this effect is mostly due to the difficulty in naming similar colors. Color use for color with a basic color term is affected only when the colors of target and distractors are very similar. The salience of our alternative attribute size, manipulated by varying the difference in size between target and distractors, had no impact on the use of color. (shrink)

It has often been observed that color is a highly preferred attribute for use in distinguishing descriptions, that is, referring expressions produced with the purpose of identifying an object within a visual scene. However, most of these observations were based on visual displays containing only colors that were maximally different in hue and for which the language of experimentation possessed basic color terms. The experiments described in this paper investigate whether speakers’ preference for color is reduced if the color of (...) the target referent is similar to that of the distractors. Because colors that look similar are often also harder to distinguish linguistically, we also examine the impact of the codability of color values. As a third factor, we investigate the salience of available alternative attributes and its impact on the use of color. The results of our experiments show that, while speakers are indeed less likely to use color when the colors in a display are similar, this effect is mostly due to the difficulty in naming similar colors. Color use for color with a basic color term is affected only when the colors of target and distractors are very similar (yet still distinguishable). The salience of our alternative attribute size, manipulated by varying the difference in size between target and distractors, had no impact on the use of color. (shrink)