Two visual world experiments investigated the processing of the implicature associated with some using a “gumball paradigm.” On each trial, participants saw an image of a gumball machine with an upper chamber with orange and blue gumballs and an empty lower chamber. Gumballs dropped to the lower chamber, creating a contrast between a partitioned set of gumballs of one color and an unpartitioned set of the other. Participants then evaluated spoken statements, such as “You got some of the (...) blue gumballs.” Experiment 1 investigated the time course of the pragmatic enrichment from some to not all when the only utterance alternatives available to refer to the different sets were some and all. In Experiment 2, the number terms two, three, four, and five were also included in the set of alternatives. Scalar implicatures were delayed relative to the interpretation of literal statements with all only when number terms were available. The results are interpreted as evidence for a constraint-based account of scalar implicature processing. (shrink)

We investigated the effects of everyday language exposure on the prediction of orthographic and phonological forms of a highly predictable word during listening comprehension. Native Japanese speakers in Tokyo (Experiment 1) and Berlin (Experiment 2) listened to sentences that contained a predictable word and viewed four objects. The critical object represented the target word (e.g., /sakana/;fish), an orthographic competitor (e.g., /tuno/;horn), a phonological competitor (e.g., /sakura/;cherry blossom), or an unrelated word (e.g., /hon/;book). The three other objects were distractors. The Tokyo (...) group fixated the target and the orthographic competitor over the unrelated objects before the target word was mentioned, suggesting that they pre-activated the orthographic form of the target word. The Berlin group showed a weaker bias toward the target than the Tokyo group, and they showed a tendency to fixate the orthographic competitor only when the orthographic similarity was very high. Thus, prediction effects were weaker in the Berlin group than in the Tokyo group. We found no evidence for the prediction of phonological information. The obtained group differences support probabilistic models of prediction, which regard the built-up language experience as a basis of prediction. (shrink)

A number of studies in the extant literature report findings that suggest asymmetry in the way reflexive and pronoun anaphors are interpreted in the early stages of processing: that pronouns are less sensitive to structural constraints, as formulated by Binding Theory, than reflexives, in the initial antecedent retrieval process. However, in previous visual world paradigm eye-tracking studies, these conclusions were based on sentences that placed the critical anaphors within picture noun phrases or prepositional phrases, which have independently (...) been shown not to neatly conform to the Binding Theory principles. We present results from a visual world paradigm eye-tracking experiment that show that when critical anaphors are placed in the indirect object position immediately following a verb as a recipient argument, pronoun and reflexive processing are equally sensitive to structural constraints. (shrink)

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the (...) class='Hi'>visual context impinge minutely on a dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP. (shrink)

The visual behaviour of consumers buying products in a supermarket was measured and used to analyse the stages of their decision process. Traditionally metrics used to trace decision-making processes are difficult to use in natural environments that often contain many options and unstructured information. Unlike previous attempts in this direction, our methodology reveals differences between a decision-making task and a search task. In particular the second stage of a decision task contains more re-dwells than the second stage of a (...) comparable search task. This study addresses the growing concern of taking eye movement research from the laboratory into the ‘real-world’, so findings can be better generalised to natural situations. (shrink)

We present a visual world eye-tracking study on Tseltal (a Mayan language) and investigate whether verbal information can be used to anticipate an upcoming referent. Basic word order in transitive sentences in Tseltal is Verb–Object–Subject (VOS). The verb is usually encountered first, making argument structure and syntactic information available at the outset, which should facilitate anticipation of the post-verbal arguments. Tseltal speakers listened to verb-initial sentences with either an object-predictive verb (e.g., “eat”) or a general verb (e.g., (...) “look for”) (e.g., “Ya slo’/sle ta stukel on te kereme,” Is eating/is looking (for) by himself the avocado the boy/ “The boy is eating/is looking (for) an avocado by himself”) while seeing a visual display showing one potential referent (e.g., avocado) and three distractors (e.g., bag, toy car, coffee grinder). We manipulated verb type (predictive vs. general) and recorded participants' eye movements while they listened and inspected the visual scene. Participants’ fixations to the target referent were analyzed using multilevel logistic regression models. Shortly after hearing the predictive verb, participants fixated the target object before it was mentioned. In contrast, when the verb was general, fixations to the target only started to increase once the object was heard. Our results suggest that Tseltal hearers pre-activate semantic features of the grammatical object prior to its linguistic expression. This provides evidence from a verb-initial language for online incremental semantic interpretation and anticipatory processing during language comprehension. These processes are comparable to the ones identified in subject-initial languages, which is consistent with the notion that different languages follow similar universal processing principles. (shrink)

The aim of this study was to find out how people process the dialectal variation encountered in the daily linguistic input. We conducted an eye tracking study (Visual Word Paradigm) that targeted the predictive processing of grammatical gender markers. Three different groups of Norwegian speakers took part in the experiment: one group of students from the capital Oslo, and two groups of dialect speakers from the Western Norwegian town Sogndal. One Sogndal group was defined as ``stable dialect speakers'', (...) and one as ``unstable dialect speakers'', based on a background questionnaire. The students participated in two eye tracking experiments each, one conducted in the their own dialect, and one in the other dialect (i.e., Sogn dialect for the Oslo students, and Oslo dialect for the Sogndal students). The gender systems in the two dialects differ: the Sogn dialect makes an obligatory three-gender split (Masculine, Feminine and Neuter) whereas the Oslo dialect only obligatorily makes a two gender distinction. The research question was whether speakers could use gender markers to predict the upcoming target noun in both local and non-local dialect mode, and furthermore, if they correctly could adjust their expectations based on dialect mode. The results showed that the Sogn speakers could predict upcoming linguistic material both in the local and Oslo dialect, but only the stable group were able to adjust their predictions based on the dialect mode. The unstable group applied a more general Oslo-compatible parsing to both the local and the non-local dialect. The Oslo speakers on the other hand were able to use gender markers as predictors only in their own dialect. We argue that the stable Sogn group should be treated as a bilingual group, as they show native-like skills in both varieties, while the unstable Sogn group can be seen as accommodated monolinguals, in that they treat the two varieties as sharing an underspecified grammar. The Oslo group on the other hand lacks sufficient competence in the other dialect to make use of grammatical markers to make predictions. (shrink)

Difficulties understanding speech form one of the most prevalent complaints among older adults. Successful speech perception depends on top-down linguistic and cognitive processes that interact with the bottom-up sensory processing of the incoming acoustic information. The relative roles of these processes in age-related difficulties in speech perception, especially when listening conditions are not ideal, are still unclear. In the current study, we asked whether older adults with a larger working memory capacity process speech more efficiently than peers with lower capacity (...) when speech is presented in noise, with another task performed in tandem. Using the Eye-tracking of Word Identification in Noise Under Memory Increased Load an adapted version of the “visual world” paradigm, 36 older listeners were asked to follow spoken instructions presented in background noise, while retaining digits for later recall under low or high memory load. In critical trials, instructions directed listeners’ gaze to pictures of objects whose names shared onset or offset sounds with the name of a competitor that was displayed on the screen at the same time. We compared listeners with different memory capacities on the time course for spoken word recognition under the two memory loads by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results indicated two trends. For older adults with lower working memory capacity, increased memory load did not affect online speech processing, however, it impaired offline word recognition accuracy. The reverse pattern was observed for older adults with higher working memory capacity: increased task difficulty significantly decreases online speech processing efficiency but had no effect on offline word recognition accuracy. Results suggest that in older adults, adaptation to adverse listening conditions is at least partially supported by cognitive reserve. Therefore, additional cognitive capacity may lead to greater resilience of older listeners to adverse listening conditions. The differential effects documented by eye movements and accuracy highlight the importance of using both online and offline measures of speech processing to explore age-related changes in speech perception. (shrink)

This study investigated the relative role of sub-syllabic components in spoken word recognition of Mandarin Chinese using an eye-tracking experiment with a visual world paradigm. Native Mandarin speakers were presented with four pictures and an auditory stimulus. They were required to click the picture according to the sound stimulus they heard, and their eye movements were tracked during this process. For a target word, nine conditions of competitors were constructed in terms of the amount of their phonological (...) overlap with the target: consonant competitor, rime competitor, tone competitor, consonant plus rime competitor, consonant plus tone competitor, rime plus tone competitor, cohort competitor, cohort plus tone competitor, and baseline competitor. A growth curve analysis was conducted with the fixation to competitors, targets, and distractors, and the results showed that competitors with consonant or rime overlap can be adequately activated, while tone overlap plays a weaker role since additional tonal information can strengthen the competitive effect only when it was added to a candidate that already bears much phonological similarity with the target. Mandarin words are processed in an incremental way in the time course of word recognition since different partially overlapping competitors could be activated immediately; like the pattern found in English, both cohort and rime competitors were activated to compete for lexical activation, but these two competitors were not temporally distinctive and mainly differed in the size of their competitive effects. Generally, the gradation of activation based on the phonological similarity between target and candidates found in this study was in line with the continuous mapping models and may reflect a strategy of native speakers shaped by the informative characteristics of the interaction among different sub-syllabic components. (shrink)

This study investigated the perception of Mandarin tonal alternations in disyllabic words. In Mandarin, a low-dipping Tone3 is converted to a high-rising Tone2 when followed by another Tone3, known as third tone sandhi. Although previous studies showed statistically significant differences in F0 between a high-rising Sandhi-Tone3 and a Tone2, native Mandarin listeners failed to correctly categorize these two tones in perception tasks. The current study utilized the visual-world paradigm in eye-tracking to further examine whether acoustic details in (...) lexical tone aid lexical access in Mandarin. Results showed that Mandarin listeners tend to process Tone2 as Tone2 whereas they tend to first process Sandhi-T3 as both Tone3 and Tone2, then later detect the acoustic differences between the two tones revealed by the sandhi context, and finally activate the target word during lexical access. The eye-tracking results suggest that subtle acoustic details of F0 may facilitate lexical access in automatic fashion in a tone language. (shrink)

Referential gaze has been shown to benefit language processing in situated communication in terms of shifting visual attention and leading to shorter reaction times on subsequent tasks. The present study simultaneously assessed both visual attention and, importantly, the immediate cognitive load induced at different stages of sentence processing. We aimed to examine the dynamics of combining visual and linguistic information in creating anticipation for a specific object and the effect this has on language processing. We report evidence (...) from three visual‐world eye‐tracking experiments, showing that referential gaze leads to a shift in visual attention toward the cued object, which consequently lowers the effort required for processing the linguistic reference. Importantly, perceiving and following the gaze cue did not prove costly in terms of cognitive effort, unless the cued object did not fit the verb selectional preferences. (shrink)

The present “visual world” eye-tracking study examined the time-course of how native and non-native speakers keep track of implied object-state representations during real-time language processing. Fifty-two native speakers of English and 46 non-native speakers with advanced English proficiency joined this study. They heard short stories describing a target object either having undergone a substantial change-of-state or a minimal change-of-state while their eye movements toward competing object-states and two unrelated distractors were tracked. We found that both groups successfully (...) directed their visual attention toward the end-state of the target object that was implied in the linguistic context. However, neither group showed anticipatory eye movements toward the implied object-state when hearing the critical verb. Only native English speakers but not non-native speakers showed a bias in visual attention during the determiner before the noun. Our results suggested that although native and non-native speakers of English largely overlapped in their time-courses of keeping track of object-state representations during real-time language comprehension, non-native speakers showed a short delay in updating the implied object-state representations. (shrink)

The human sentence processor is able to make rapid predictions about upcoming linguistic input. For example, upon hearing the verb eat, anticipatory eye-movements are launched toward edible objects in a visual scene. However, the cognitive mechanisms that underlie anticipation remain to be elucidated in ecologically valid contexts. Previous research has, in fact, mainly used clip-art scenes and object arrays, raising the possibility that anticipatory eye-movements are limited to displays containing a small number of objects in a visually impoverished context. (...) In Experiment 1, we confirm that anticipation effects occur in real-world scenes and investigate the mechanisms that underlie such anticipation. In particular, we demonstrate that real-world scenes provide contextual information that anticipation can draw on: When the target object is not present in the scene, participants infer and fixate regions that are contextually appropriate. Experiment 2 investigates whether such contextual inference requires the co-presence of the scene, or whether memory representations can be utilized instead. The same real-world scenes as in Experiment 1 are presented to participants, but the scene disappears before the sentence is heard. We find that anticipation occurs even when the screen is blank, including when contextual inference is required. We conclude that anticipatory language processing is able to draw upon global scene representations to make contextual inferences. These findings are compatible with theories assuming contextual guidance, but posit a challenge for theories assuming object-based visual indices. (shrink)

Pronoun interpretation is often described as relying on a comprehender's mental model of discourse. For example, in some psycholinguistic accounts, interpreting pronouns involves a process of retrieval, whereby a pronoun is resolved by accessing information from its linguistic antecedent. However, linguistic antecedents are neither necessary nor sufficient for interpreting a pronoun, and even when an antecedent has been introduced in earlier discourse, there is little evidence for the retrieval of linguistic form. The current study extends our understanding of pronoun interpretation (...) by examining whether the semantics of antecedent expressions are retrieved from representations of past discourse. Participants were instructed to move displayed objects in a Visual World eye-tracking task. In some cases, the semantics of the antecedent were no longer viable after an instruction was completed (e.g., “Move the house on the left to area 12,” where the result was that a different house is now the leftmost one). In this case, retrieving antecedent semantics at the point of hearing a subsequent pronoun (“Now, move it…”) should entail a processing penalty. Instead, the results showed that antecedent semantics have no direct effect on interpretation, raising additional questions about the role that retrieval might play in pronoun interpretation. (shrink)

A classic problem in cognitive science concerns how listeners perceive and understand speech as comprised of discrete words. We propose a Syllable Inference account of spoken word recognition and segmentation, under which alternative hierarchical models of syllables, words, and phonemes are dynamically posited from cues that include current and past speech rate, with a goal of maximal prediction of sensory input. Three experiments using the Visual World eye‐tracking paradigm provide evidence supporting our proposal.

We investigated how two cues to contrast—beat gesture and contrastive pitch accenting—affect comprehenders' cognitive load during processing of spoken referring expressions. In two visual‐world experiments, we orthogonally manipulated the presence of these cues and their felicity, or fit, with the local (sentence‐level) referential context in critical referring expressions while comprehenders' task‐evoked pupillary responses (TEPRs) were examined. In Experiment 1, beat gesture and contrastive accenting always matched the referential context of filler referring expressions and were therefore relatively felicitous on (...) the global (experiment) level, whereas in Experiment 2, beat gesture and contrastive accenting never fit the referential context of filler referring expressions and were therefore infelicitous on the global level. The results revealed that both beat gesture and contrastive accenting increased comprehenders' cognitive load. For beat gesture, this increase in cognitive load was driven by both local and global infelicity. For contrastive accenting, this increase in cognitive load was unaffected when cues were globally felicitous but exacerbated when cues were globally infelicitous. Together, these results suggest that comprehenders' cognitive resources are taxed by processing infelicitous use of beat gesture and contrastive accenting to convey contrast on both the local and global levels. (shrink)

Access to communication is critical for individuals with late-stage amyotrophic lateral sclerosis and minimal volitional movement, but they sometimes present with concomitant visual or ocular motility impairments that affect their performance with eye tracking or visual brain-computer interface systems. In this study, we explored the use of modified eye tracking and steady state visual evoked potential BCI, in combination with the Shuffle Speller typing interface, for this population. Two participants with late-stage ALS, visual impairments, (...) and minimal volitional movement completed a single-case experimental research design comparing copy-spelling performance with three different typing systems: commercially available eye tracking communication software, Shuffle Speller with modified eye tracking, and Shuffle Speller with SSVEP BCI. Participant 1 was unable to type any correct characters with the commercial system, but achieved accuracies of up to 50% with Shuffle Speller eye tracking and 89% with Shuffle Speller BCI. Participant 2 also had higher maximum accuracies with Shuffle Speller, typing with up to 63% accuracy with eye tracking and 100% accuracy with BCI. However, participants’ typing accuracy for both Shuffle Speller conditions was highly variable, particularly in the BCI condition. Both the Shuffle Speller interface and SSVEP BCI input show promise for improving typing performance for people with late-stage ALS. Further development of innovative BCI systems for this population is needed. (shrink)

Background: Facial attractiveness influences our perceptions of others, with beautiful faces reaping societal rewards and anomalous faces encountering penalties. The purpose of this study was to determine associations of visual attention with bias and social dispositions toward people with facial anomalies. -/- Methods: Sixty subjects completed tests evaluating implicit bias, explicit bias, and social dispositions before viewing publicly available images of preoperative and postoperative patients with hemifacial microsomia. Eye-tracking was used to register visual fixations. -/- Results: Participants (...) with higher implicit bias scores fixated significantly less on the cheek and ear region preoperatively (P = 0.004). Participants with higher scores in empathic concern and perspective taking fixated more on the forehead and orbit preoperatively (P = 0.045) and nose and lips (P = 0.027) preoperativel. -/- Conclusions: Participants with higher levels of implicit bias spent less visual attention on anomalous facial anatomy, whereas participants with higher levels of empathic concern and perspective taking spent more visual attention on normal facial anatomy. Levels of bias and social dispositions such as empathy may predict layperson gaze patterns toward those with facial anomalies and provide insights to neural mechanisms underlying the "anomalous is bad" paradigm. (shrink)

At the core of Pickering & Garrod's (P&G's) theory is a monitor that uses forward models. I argue that this account is challenged by neuropsychological findings and visual world eye-tracking data and that it has two conceptual problems. I propose that conflict monitoring avoids these issues and should be considered a promising alternative to perceptual loop and forward modeling theories.

Semantic studies on diagrammatic notations (Barwise & Etchemendy, ; Shimojima, ; Stenning & Lemon, ) have revealed that the “non-deductive,” “emergent,” or “perceptual” effects of diagrams (Chandrasekaran, Kurup, Banerjee, Josephson, & Winkler, ; Kulpa, ; Larkin & Simon, ; Lindsay, ) are all rooted in the exploitation of spatial constraints on graphical structures. Thus, theoretically, this process is a key factor in inference with diagrams, explaining the frequently observed reduction of inferential load. The purpose of this study was to examine (...) the empirical basis for this theoretical suggestion, focusing on the reality of the constraint-exploitation strategy in actual practices of diagrammatic reasoning. Eye movements were recorded while participants used simple position diagrams to solve three- or four-term transitive inference problems. Our experiments revealed that the participants could exploit spatial constraints on graphical structures even when (a) they were not in the position of actually manipulating diagrams, (b) the semantic rule for the provided diagrams did not match their preferences, and (c) the constraint-exploitation strategy invited a partly adverse effect. These findings indicate that the hypothesized process is in fact robust, with the potential to broadly account for the inferential advantage of diagrams. (shrink)

This study investigates the online process of reading and analyzing of sketchnotes (visual notes containing a handwritten text and drawings) on Russian language material. Using the eye-tracking method, we compared the processing of different types of sketchnotes (‘path’ (trajectory), linear, and radial) and the processing of a verbal text. Biographies of Russian writers were used as the material. In a preliminary experiment, we asked 89 college students to read the biographies and to evaluate each text or sketch using (...) five scales (from -2 to +2). The best example for each of three formats of sketchnotes and a verbal text were chosen. In the main experiment, twenty-one secondary school students examined four different biographies in four different formats (three sketchnotes and a verbal text), answered to the factual and analytical questions to these texts and estimated the difficulty of each text. We measured the total dwell time, the total fixation count, the average fixation duration for each stimulus as well as for separate zones inside the sketches including verbal and non-verbal information. Our results show that readers process the information better and faster while reading sketchnotes than a verbal text. In the trajectory sketchnotes, the readers followed the order of elements aimed by the author of the sketchnotes better than in the radial and linear sketchnotes. The analysis of participants’ eye movements while processing the stimuli made it possible to propose several recommendations for creating effective sketchnotes. (shrink)

This chapter examines the role of the situation model in referential processing and how it can link what appear to be incompatible results from studies of monologue and dialogue as well as studies of reading and visual-world eye tracking. It shows that data from experiments on pronoun resolution in reading indicate a two-step model, in which candidate antecedents for an anaphor are first identified on the basis of gender matching and number matching, then evaluated with respect to (...) the overall situation model. In similar visual-world studies, these stages are seemingly collapsed together, which can be explained in terms of differences in how the situation model is related to the text versus the visual display of potential referents. The chapter suggests that comparing these different types of studies can elucidate the processes of reference resolution. (shrink)

In this study, 44 late elementary students’ visual behavior patterns when reading mind maps were investigated, more particularly, the intuitive processing nature of their visual characteristics, reading sequence and presentation mode. Eye-tracked data were investigated by means of static early attention and dynamic educational process mining analysis and combined with learning performance and retrospective interview data. All students seem to struggle with the map’s radial structure during initial reading. Also, the picture’s position in the map diverts students from (...) consecutively reading interconnected branches. EPM analyses revealed different reading patterns in proceeding reading behavior. Students receiving a text first, seem to grasp the radial structure slightly more and show higher information integration attempts. They also attained higher free recall and coherence scores. The study concludes with instructional design principles for urgently needed explicit visual literacy instruction in elementary grades. (shrink)

As Macnamara (1978) once asked, how can we talk about what we see? We report on a study manipulating realistic dynamic scenes and sentences aiming to understand the interaction between linguistic and visual representations in real-world situations. Specifically, we monitored participants’ eye movements as they watched video clips of everyday scenes while listening to sentences describing these scenes. We manipulated two main variables. The first was the semantic class of the verb in the sentence and the second was (...) the action/motion of the agent in the unfolding event. The sentences employed two verb classes—causatives (e.g., break) and perception/psychological (e.g., notice)—which impose different constraints on the nouns that serve as their grammatical complements. The scenes depicted events in which agents either moved towards a target object (always the referent of the verb-complement noun), away from it, or remained neutral performing a given activity (such as cooking). Scenes and sentences were synchronized such that the verb onset corresponded to the first video frame of the agent motion towards or away from the object. Results show effects of agent motion but weak verb-semantic restrictions: causatives draw more attention to potential referents of their grammatical complements than perception verbs only when the agent moves towards the target object. Crucially, we found no anticipatory verb-driven eye movements toward the target object, contrary to studies using non-naturalistic and static scenes. We propose a model in which linguistic and visual computations in real-world situations occur largely independent of each other during the early moments of perceptual input, but rapidly interact at a central, conceptual system using a common, propositional code. Implications for language use in real world contexts are discussed. (shrink)

We report the results of one visual‐world eye‐tracking experiment and two referent selection tasks in which we investigated the effects of information structure in the form of prosody and word order manipulation on the processing of subject pronouns er and der in German. Factors such as subjecthood, focus, and topicality, as well as order of mention have been linked to an increased probability of certain referents being selected as the pronoun's antecedent and described as increasing this referent's (...) prominence, salience, or accessibility. The goal of this study was to find out whether pronoun processing is primarily guided by linguistic factors (e.g., grammatical role) or nonlinguistic factors (e.g., first‐mention), and whether pronoun interpretation can be described in terms of referents' “prominence” / “accessibility” / “salience.” The results showed an overall subject preference for er, whereas der was affected by the object role and focus marking. While focus increases the attentional load and enhances memory representation for the focused referent making the focused referent more available, ultimately it did not affect the final interpretation of er, suggesting that “prominence” or the related concepts do not explain referent selection preferences. Overall, the results suggest a primacy of linguistic factors in determining pronoun resolution. (shrink)

Online data collection with infants raises special opportunities and challenges for developmental research. One of the most prevalent methods in infancy research is eye-tracking, which has been widely applied in laboratory settings to assess cognitive development. Technological advances now allow conducting eye-tracking online with various populations, including infants. However, the accuracy and reliability of online infant eye-tracking remain to be comprehensively evaluated. No research to date has directly compared webcam-based and in-lab eye-tracking data from infants, similarly (...) to data from adults. The present study provides a direct comparison of in-lab and webcam-based eye-tracking data from infants who completed an identical looking time paradigm in two different settings. We assessed 4-6-month-old infants in an eye-tracking task that measured the detection of audio-visual asynchrony. Webcam-based and in-lab eye-tracking data were compared on eye-tracking and video data quality, infants’ viewing behavior, and experimental effects. Results revealed no differences between the in-lab and online setting in the frequency of technical issues and participant attrition rates. Video data quality was comparable between settings in terms of completeness and brightness, despite lower frame rate and resolution online. Eye-tracking data quality was higher in the laboratory than online, except in case of relative sample loss. Gaze data quantity recorded by eye-tracking was significantly lower than by video in both settings. In valid trials, eye-tracking and video data captured infants’ viewing behavior uniformly, irrespective of setting. Despite the common challenges of infant eye-tracking across experimental settings, our results point toward the necessity to further improve the precision of online eye-tracking with infants. Taken together, online eye-tracking is a promising tool to assess infants’ gaze behavior but requires careful data quality control. The demographic composition of both samples differed from the generic population on caregiver education: our samples comprised caregivers with higher-than-average education levels, challenging the notion that online studies will per se reach more diverse populations. (shrink)

We adopt the visual-world eye-tracking paradigm to test the hypothesis that scalar implicatures are integrated very locally to the utterance of scalar terms. Focusing on the and,or scale, we show that early point-of-disambiguation effects similar to those triggered by the integration of the lexical meaning of and can be triggered by the integration of the exhaustive meaning of or. Some design issues and an independent interpretive asymmetry holding between and and or are discussed as possible explanations for (...) remaining differences between the effects found in the two cases. We conclude that the exclusivity implicature that is associated to sentences containing or seems to be calculated online, rather locally to the utterance of the disjunction. (shrink)

Previous processing studies have shown that constituents that are prosodically marked as focus lead to an activation of alternatives. We investigate the processing of constituents that are prosodically marked as contrastive topics. In German, contrastive topics are prosodically realized by prenuclear L*+H accents. Our study tests a) whether prenuclear accents (as opposed to nuclear accents) are able to activate contrastive alternatives, b) whether they do this in the same way as constituents prosodically marked as focus with nuclear accents do, which (...) is important for semantic modeling and c) whether the activation of alternatives is caused by pitch accent type (prenuclear L*+H as contrastive accent vs. prenuclear L+H* as non-contrastive accent) or by differences in F0-excursion (related to prominence). We conducted two visual-world eye-tracking studies, in which German listeners heard declarative utterances (e.g., The swimmer wanted to put on flappers) and watched displays that depicted four printed words: one that was a contrastive alternative to the subject noun (e.g., diver), one that was non-contrastively related to it (e.g., sports), the object (e.g., flappers), which had to be clicked, and an unrelated distractor. Experiment 1 presented participants with two naturally produced intonation conditions, a broad focus control condition with a prenuclear L+H* accent on the subject and a contrastive topic condition with a prenuclear L*+H accent. The results showed that participants fixated more on the contrastive alternative when the subject was produced with an L*+H accent, with the same effect size and timing as reported for focus constituents. Experiment 2 resynthesized the stimuli so that peak height and F0-excursion were the same across intonation conditions. The effect was the same, but the time course was slightly later. Our results suggest that prenuclear L*+H immediately leads to the activation of alternatives during online processing, and that the F0-excursion of the accent lends little. The results are discussed with regard to the processing of contrastive focus accents and theories of contrastive topic. (shrink)

Previous research has shown that language comprehenders resolve reference quickly and incrementally, but not much is known about the neural processes and representations that are involved. Studies of visual short-term memory suggest that access to the representation of an item from a previously seen display is associated with a negative evoked potential at posterior electrodes contralateral to the spatial location of that item in the display. In this paper we demonstrate that resolving the reference of a noun phrase in (...) a recently seen visual display is associated with an event-related potential that is analogous to this effect. Our design was adapted from the visual world paradigm: in each trial, participants saw a display containing three simple objects, followed by a question about the objects, such as Was the pink fish next to a boat?, presented word by word. Questions differed in whether the color adjective allowed the reader to identify the referent of the noun phrase or not (i.e., whether one or more objects of the named color were present). Consistent with our hypothesis, we observed that reference resolution by the adjective was associated with a negative evoked potential at posterior electrodes contralateral to spatial location of the referent, starting approximately 333 ms after the onset of the adjective. The fact that the laterality of the effect depended upon the location of the referent within the display suggests that reference resolution in visual domains involves, at some level, a modality-specific representation. In addition, the effect gives us an estimate of the time course of processing from perception of the written word to the point at which its meaning is brought into correspondence with the referential domain. (shrink)

The theoretical notion of ‘construal’ captures the idea that the way in which we describe a scene reflects our conceptualization of it. Relying on the concept of ception – which conjoins conception and perception – we operationalized construal and employed a Visual World Paradigm to establish which aspects of linguistic scene description modulate visual scene perception, thereby affecting event conception. By analysing viewing behaviour after alternating ways of describing location (prepositions), agentivity (active/passive voice) and transfer (NP/pp datives), (...) we found that the linguistic construal of a scene affects its spontaneous visual perception in two ways: either by determining the order in which the components of a scene are accessed or by modulating the distribution of attention over the components, making them more or less salient than they naturally are. We also found evidence for the existence of a cline in the construal effect with stronger expressive differences, such as the prepositional manipulation, inducing more prominent changes in visual perception than the dative manipulation. We discuss the claims language can lay to affecting visual information uptake and hence conceptualization of a static scene in the light of these results. (shrink)