Online data collection with infants raises special opportunities and challenges for developmental research. One of the most prevalent methods in infancy research is eye-tracking, which has been widely applied in laboratory settings to assess cognitive development. Technological advances now allow conducting eye-tracking online with various populations, including infants. However, the accuracy and reliability of online infant eye-tracking remain to be comprehensively evaluated. No research to date has directly compared webcam-based and in-lab eye-tracking data (...) from infants, similarly to data from adults. The present study provides a direct comparison of in-lab and webcam-based eye-tracking data from infants who completed an identical looking time paradigm in two different settings. We assessed 4-6-month-old infants in an eye-tracking task that measured the detection of audio-visual asynchrony. Webcam-based and in-lab eye-tracking data were compared on eye-tracking and video data quality, infants’ viewing behavior, and experimental effects. Results revealed no differences between the in-lab and online setting in the frequency of technical issues and participant attrition rates. Video data quality was comparable between settings in terms of completeness and brightness, despite lower frame rate and resolution online. Eye-tracking data quality was higher in the laboratory than online, except in case of relative sample loss. Gaze data quantity recorded by eye-tracking was significantly lower than by video in both settings. In valid trials, eye-tracking and video data captured infants’ viewing behavior uniformly, irrespective of setting. Despite the common challenges of infant eye-tracking across experimental settings, our results point toward the necessity to further improve the precision of online eye-tracking with infants. Taken together, online eye-tracking is a promising tool to assess infants’ gaze behavior but requires careful data quality control. The demographic composition of both samples differed from the generic population on caregiver education: our samples comprised caregivers with higher-than-average education levels, challenging the notion that online studies will per se reach more diverse populations. (shrink)

This study investigates the online process of reading and analyzing of sketchnotes (visual notes containing a handwritten text and drawings) on Russian language material. Using the eye-tracking method, we compared the processing of different types of sketchnotes (‘path’ (trajectory), linear, and radial) and the processing of a verbal text. Biographies of Russian writers were used as the material. In a preliminary experiment, we asked 89 college students to read the biographies and to evaluate each text or sketch using (...) five scales (from -2 to +2). The best example for each of three formats of sketchnotes and a verbal text were chosen. In the main experiment, twenty-one secondary school students examined four different biographies in four different formats (three sketchnotes and a verbal text), answered to the factual and analytical questions to these texts and estimated the difficulty of each text. We measured the total dwell time, the total fixation count, the average fixation duration for each stimulus as well as for separate zones inside the sketches including verbal and non-verbal information. Our results show that readers process the information better and faster while reading sketchnotes than a verbal text. In the trajectory sketchnotes, the readers followed the order of elements aimed by the author of the sketchnotes better than in the radial and linear sketchnotes. The analysis of participants’ eye movements while processing the stimuli made it possible to propose several recommendations for creating effective sketchnotes. (shrink)

This study investigated the impact of online product reviews on consumers purchasing decisions by using eye-tracking. The research methodology involved development of a conceptual framework of online product review and purchasing intention through the moderation role of gender and visual attention in comments, and empirical investigation into the region of interest analysis of consumers fixation during the purchase decision process and behavioral analysis. The results showed that consumers’ attention to negative comments was significantly greater than that to (...) positive comments, especially for female consumers. Furthermore, the study identified a significant correlation between the visual browsing behavior of consumers and their purchase intention. It also found that consumers were not able to identify false comments. The current study provides a deep understanding of the underlying mechanism of how online reviews influence shopping behavior, reveals the effect of gender on this effect for the first time and explains it from the perspective of attentional bias, which is essential for the theory of online consumer behavior. Specifically, the different effects of consumers’ attention to negative comments seem to be moderated through gender with female consumers’ attention to negative comments being significantly greater than to positive ones. These findings suggest that practitioners need to pay particular attention to negative comments and resolve them promptly through the customization of product/service information, taking into consideration consumer characteristics, including gender. (shrink)

This article argues that a theory of media selectivity needs a theory of attention, because attention to a media stimulus is the starting point of each process of reception. Attention sequences towards media stimuli – pages of newspapers and online-newspapers – were analyzed using eye-tracking patterns from three different perspectives. First, attention patterns were compared under varying task conditions. Second, different types of media were tested. Third, attention sequences towards different forms of news with different design patterns were (...) compared. Attention was seen as a prerequisite for reception: Its selective functions for these processes are especially important. Reception itself was examined within an action-theoretical framework and therefore described as a form of interaction between recipient and the media. Eye-tracking data were used as indicators of attention. Starting with a hypothesis on the impact of different media such as printed newspapers and online newspapers on the agenda-setting process of their audience, the study examined how the type of media and the form of news influences attention and selectivity. Our findings showed that visual cues such as salient photos or graphics and information hierarchies signalled by design and layout guide attention processes, not as an automatic process driven from the bottom up, but as stimuli for an active, intention-driven selection process. The results indicate that the form of news affects these patterns of interactive attention more than the medium itself. (shrink)

Difficulties understanding speech form one of the most prevalent complaints among older adults. Successful speech perception depends on top-down linguistic and cognitive processes that interact with the bottom-up sensory processing of the incoming acoustic information. The relative roles of these processes in age-related difficulties in speech perception, especially when listening conditions are not ideal, are still unclear. In the current study, we asked whether older adults with a larger working memory capacity process speech more efficiently than peers with lower capacity (...) when speech is presented in noise, with another task performed in tandem. Using the Eye-tracking of Word Identification in Noise Under Memory Increased Load an adapted version of the “visual world” paradigm, 36 older listeners were asked to follow spoken instructions presented in background noise, while retaining digits for later recall under low or high memory load. In critical trials, instructions directed listeners’ gaze to pictures of objects whose names shared onset or offset sounds with the name of a competitor that was displayed on the screen at the same time. We compared listeners with different memory capacities on the time course for spoken word recognition under the two memory loads by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results indicated two trends. For older adults with lower working memory capacity, increased memory load did not affect online speech processing, however, it impaired offline word recognition accuracy. The reverse pattern was observed for older adults with higher working memory capacity: increased task difficulty significantly decreases online speech processing efficiency but had no effect on offline word recognition accuracy. Results suggest that in older adults, adaptation to adverse listening conditions is at least partially supported by cognitive reserve. Therefore, additional cognitive capacity may lead to greater resilience of older listeners to adverse listening conditions. The differential effects documented by eye movements and accuracy highlight the importance of using both online and offline measures of speech processing to explore age-related changes in speech perception. (shrink)

Modern consumers face a dramatic rise in web-based technological advancements and have trouble making rational and proper decisions when they shop online. When they try to make decisions about products and services, they also feel pressured against time when sorting among all of the unnecessary items in the flood of information available on the web. In this sense, they need to use consumer decision-making creativity to make rational decisions. However, unexplored research questions on this subject remain. First, in what (...) ways do task difficulty and time constraints affect visual attention on exploitative and exploratory activities differently? Second, how does the location of the reference influence the level of visual attention to exploitative and exploratory activities depending on affordance theory? Third, how do exploratory and exploitative activities affect CDMC? Eye-tracking experiments were conducted with 70 participants to obtain relevant metrics such as total fixation duration, fixation count, and visit count to answer these research questions. Our findings suggest that task difficulty influences exploitative activity, whereas time constraint is related to the exploratory activity. The result of the location of hints aligns with the affordance theory for the exploitative activity. Besides, exploratory activity positively affected CDMC, but exploitative activity did not show any effect. (shrink)

We present a visual world eye-tracking study on Tseltal (a Mayan language) and investigate whether verbal information can be used to anticipate an upcoming referent. Basic word order in transitive sentences in Tseltal is Verb–Object–Subject (VOS). The verb is usually encountered first, making argument structure and syntactic information available at the outset, which should facilitate anticipation of the post-verbal arguments. Tseltal speakers listened to verb-initial sentences with either an object-predictive verb (e.g., “eat”) or a general verb (e.g., “look for”) (...) (e.g., “Ya slo’/sle ta stukel on te kereme,” Is eating/is looking (for) by himself the avocado the boy/ “The boy is eating/is looking (for) an avocado by himself”) while seeing a visual display showing one potential referent (e.g., avocado) and three distractors (e.g., bag, toy car, coffee grinder). We manipulated verb type (predictive vs. general) and recorded participants' eye movements while they listened and inspected the visual scene. Participants’ fixations to the target referent were analyzed using multilevel logistic regression models. Shortly after hearing the predictive verb, participants fixated the target object before it was mentioned. In contrast, when the verb was general, fixations to the target only started to increase once the object was heard. Our results suggest that Tseltal hearers pre-activate semantic features of the grammatical object prior to its linguistic expression. This provides evidence from a verb-initial language for online incremental semantic interpretation and anticipatory processing during language comprehension. These processes are comparable to the ones identified in subject-initial languages, which is consistent with the notion that different languages follow similar universal processing principles. (shrink)

Behavioral studies on language processing rely on the eye-mind assumption, which states that the time spent looking at text is an index of the time spent processing it. In most cases, relatively shorter reading times are interpreted as evidence of greater processing efficiency. However, previous evidence from L2 research indicates that non-native participants who present fast reading times are not always more efficient readers, but rather shallow parsers. Because earlier studies did not identify a reliable predictor of variability in L2 (...) processing, such uncertainty around the interpretation of reading times introduces a potential confound that undermines the credibility and the conclusions of online measures of processing. The present study proposes that a recently developed modulator of online processing efficiency, namely, chunking ability, may account for the observed variability in L2 online reading performance. L1 English – L2 Spanish learners’ eye movements were analyzed during natural reading. Chunking ability was predictive of overall reading speed. Target relative clauses contained L2 Verb-Noun multiword units, which were manipulated with regards to their L1-L2 congruency. The results indicated that processing of the L1-L2 incongruent units was modulated by an interaction of L2 chunking ability and level of knowledge of multiword units. Critically, the data revealed an inverse U-shaped pattern, with faster reading times in both learners with the highest and the lowest chunking ability scores, suggesting fast integration in the former, and lack of integration in the latter. Additionally, the presence of significant differences between conditions was correlated with individual chunking ability. The findings point at chunking ability as a significant modulator of general L2 processing efficiency, and of cross-language differences in particular, and add clarity to the interpretation of variability in the online reading performance of non-native speakers. (shrink)

The literature on irony processing mainly focused on contextual effect, leaving other factors untouched. The current study investigated how social status information affected the online comprehension of irony. As irony might be more damaging when a speaker uses it to a superordinate than the other way around, it is assumed that greater processing efforts would be observed in the former case. Using an eye-movement sentence reading paradigm, we recruited 36 native Mandarin speakers and examined the role of social status (...) information and literality in their irony interpretation. Our results showed ironic statements were more effortful to process than literal ones, reporting an early and consistent effect on the target regions. The social status effect followed the literality effect, with more difficulty in processing ironic statements that targeted the superordinate than the subordinate; such an effect of social status was missing with literal statements. Besides, an individual’s social skill appeared to affect the perception of status information in ironic statements, as the socially skillful readers needed more time than the socially unskillful to process irony targeting a subordinate in the second half of the experiment in the critical region. Our study suggests that irony processing might be further discussed in terms of the relative predictability of linguistic, social, and individual variabilities. (shrink)

We investigated how two cues to contrast—beat gesture and contrastive pitch accenting—affect comprehenders' cognitive load during processing of spoken referring expressions. In two visual‐world experiments, we orthogonally manipulated the presence of these cues and their felicity, or fit, with the local (sentence‐level) referential context in critical referring expressions while comprehenders' task‐evoked pupillary responses (TEPRs) were examined. In Experiment 1, beat gesture and contrastive accenting always matched the referential context of filler referring expressions and were therefore relatively felicitous on the global (...) (experiment) level, whereas in Experiment 2, beat gesture and contrastive accenting never fit the referential context of filler referring expressions and were therefore infelicitous on the global level. The results revealed that both beat gesture and contrastive accenting increased comprehenders' cognitive load. For beat gesture, this increase in cognitive load was driven by both local and global infelicity. For contrastive accenting, this increase in cognitive load was unaffected when cues were globally felicitous but exacerbated when cues were globally infelicitous. Together, these results suggest that comprehenders' cognitive resources are taxed by processing infelicitous use of beat gesture and contrastive accenting to convey contrast on both the local and global levels. (shrink)

Upon hearing a scalar adjective in a definite referring expression such as “the big…,” listeners typically make anticipatory eye movements to an item in a contrast set, such as a big glass in the context of a smaller glass. Recent studies have suggested that this rapid, contrastive interpretation of scalar adjectives is malleable and calibrated to the speaker's pragmatic competence. In a series of eye‐tracking experiments, we explore the nature of the evidence necessary for the modulation of pragmatic inferences (...) in language comprehension, focusing on the complementary roles of top–down information ‐ (knowledge about the particular speaker's pragmatic competence) and bottom‐up cues (distributional information about the use of scalar adjectives in the environment). We find that bottom‐up evidence alone (e.g., the speaker says “the big dog” in a context with one dog), in large quantities, can be sufficient to trigger modulation of the listener's contrastive inferences, with or without top‐down cues to support this adaptation. Further, these findings suggest that listeners track and flexibly combine multiple sources of information in service of efficient pragmatic communication. (shrink)

Subject relative clauses (SRCs) are typically processed more easily than object relative clauses (ORCs), but this difference is diminished by an inanimate head‐noun in semantically non‐reversible ORCs (“The book that the boy is reading”). In two eye‐tracking experiments, we investigated the influence of animacy on online processing of semantically reversible SRCs and ORCs using lexically inanimate items that were perceptually animate due to motion (e.g., “Where is the tractor that the cow is chasing”). In Experiment 1, 48 children (...) (aged 4;5–6;4) and 32 adults listened to sentences that varied in the lexical animacy of the NP1 head‐noun (Animate/Inanimate) and relative clause (RC) type (SRC/ORC) with an animate NP2 while viewing two images depicting opposite actions. As expected, inanimate head‐nouns facilitated the correct interpretation of ORCs in children; however, online data revealed children were more likely to anticipate an SRC as the RC unfolded when an inanimate head‐noun was used, suggesting processing was sensitive to perceptual animacy. In Experiment 2, we repeated our design with inanimate (rather than animate) NP2s (e.g., “where is the tractor that the car is following”) to investigate whether our online findings were due to increased visual surprisal at an inanimate as agent, or to similarity‐based interference. We again found greater anticipation for an SRC in the inanimate condition, supporting our surprisal hypothesis. Across the experiments, offline measures show that lexical animacy influenced children's interpretation of ORCs, whereas online measures reveal that as RCs unfolded, children were sensitive to the perceptual animacy of lexically inanimate NPs, which was not reflected in the offline data. Overall measures of syntactic comprehension, inhibitory control, and verbal short‐term memory and working memory were not predictive of children's accuracy in RC interpretation, with the exception of a positive correlation with a standardized measure of syntactic comprehension in Experiment 1. (shrink)

Three experiments explored online recognition in a nonspeech domain, using a novel experimental paradigm. Adults learned to associate abstract shapes with particular melodies, and at test they identified a played melody’s associated shape. To implicitly measure recognition, visual fixations to the associated shape versus a distractor shape were measured as the melody played. Degree of similarity between associated melodies was varied to assess what types of pitch information adults use in recognition. Fixation and error data suggest that adults naturally (...) recognize music, like language, incrementally, computing matches to representations before melody offset, despite the fact that music, unlike language, provides no pressure to execute recognition rapidly. Further, adults use both absolute and relative pitch information in recognition. The implicit nature of the dependent measure should permit use with a range of populations to evaluate postulated developmental and evolutionary changes in pitch encoding. (shrink)

When we read narrative texts such as novels and newspaper articles, we segment information presented in such texts into discrete events, with distinct boundaries between those events. But do our eyes reflect this event structure while reading? This study examines whether eye movements during the reading of discourse reveal how readers respond online to event structure. Participants read narrative passages as we monitored their eye movements. Several measures revealed that event structure predicted eye movements. In two experiments, we found (...) that both early and overall reading times were longer for event boundaries. We also found that regressive saccades were more likely to land on event boundaries, but that readers were less likely to regress out of an event boundary. Experiment 2 also demonstrated that tracking event structure carries a working memory load. Eye movements provide a rich set of online data to test the cognitive reality of event segmentation during reading. (shrink)

In two eye-tracking studies we investigated the influence of Mandarin numeral classifiers – a grammatical category in the language – on online overt attention. Mandarin speakers were presented with simple sentences through headphones while their eye-movements to objects presented on a computer screen were monitored. The crucial question is what participants look at while listening to a pre-specified target noun. If classifier categories influence Mandarin speakers' general conceptual processing, then on hearing the target noun they should look at (...) objects that are members of the same classifier category – even when the classifier is not explicitly present. The data show that when participants heard a classifier they shifted overt attention significantly more to classifiermatch objects than to distractor objects, but when the classifier was not explicitly presented in speech, overt attention to classifier-match objects and distractor objects did not differ. This suggests that although classifier distinctions do influence eye-gaze behavior, they do so only during linguistic processing of that distinction and not in moment-to-moment general conceptual processing. (shrink)

Although the presence or absence of a pitch accent clearly can play an important role in signaling the discourse and information structure of an utterance, whether the form of an accent determines the type of information it conveys is more controversial. We used an eye‐tracking paradigm to investigate whether H*, which has been argued to signal new information, evokes different eye fixations than L+H*, which has been argued to signal the presence of contrast. Our results demonstrate that although listeners (...) interpret these accents differently, their interpretive domains overlap. L+H* creates a strong bias toward contrast referents whereas H* is compatible with both new and contrast referents. (shrink)

Semantic studies on diagrammatic notations (Barwise & Etchemendy, ; Shimojima, ; Stenning & Lemon, ) have revealed that the “non-deductive,” “emergent,” or “perceptual” effects of diagrams (Chandrasekaran, Kurup, Banerjee, Josephson, & Winkler, ; Kulpa, ; Larkin & Simon, ; Lindsay, ) are all rooted in the exploitation of spatial constraints on graphical structures. Thus, theoretically, this process is a key factor in inference with diagrams, explaining the frequently observed reduction of inferential load. The purpose of this study was to examine (...) the empirical basis for this theoretical suggestion, focusing on the reality of the constraint-exploitation strategy in actual practices of diagrammatic reasoning. Eye movements were recorded while participants used simple position diagrams to solve three- or four-term transitive inference problems. Our experiments revealed that the participants could exploit spatial constraints on graphical structures even when (a) they were not in the position of actually manipulating diagrams, (b) the semantic rule for the provided diagrams did not match their preferences, and (c) the constraint-exploitation strategy invited a partly adverse effect. These findings indicate that the hypothesized process is in fact robust, with the potential to broadly account for the inferential advantage of diagrams. (shrink)

Face perception and emotion categorization are widely investigated under laboratory conditions that are devoid of real social interaction. Using mobile eye-tracking glasses in a standardized diagnostic setting while applying the Autism Diagnostic Observation Schedule, we had the opportunity to record gaze behavior of children and adolescents with and without Autism Spectrum Conditions during social interaction. The objective was to investigate differences in eye-gaze behavior between three groups of children and adolescents either with ASC or with unconfirmed diagnosis of ASC (...) or with neurotypical development during social interaction with an adult interviewer in a diagnostic standard situation using the ADOS-2. In a case control study, we used mobile eye-tracking glasses in an ecologically valid and highly standardized diagnostic interview to investigate suspected cases of ASC. After completion of the ASC diagnostic gold standard including the ADOS-2, the participants were assigned to two groups based on their diagnosis and compared with a matched group of neurotypically developed controls. The primary outcome measure is the percentage of total dwell times assessed for different areas of interest with regard to the face and body of a diagnostic interviewer and the surrounding space. Overall, 65 children and adolescents within an age range of 8.3–17.9 years were included in the study. The data revealed significant group differences, especially in the central-face area. Previous investigations under laboratory conditions gave preferential attention to the eye region during face perception to describe differences between ASC and NTD. In this study – using an ecologically valid setting within a standard diagnostic procedure – the results indicate that neurotypically developed controls seem to process faces and facial expressions in a holistic manner originating from the central-face region. Conversely, participants on the Autism Spectrum seem to avoid the central-face region and show unsystematic gaze behavior, not using the preferred landing position in the central-face region as the Archimedean point of face perception. This study uses a new approach, and it will be important to replicate these preliminary findings in future research. (shrink)

Failure increases the motivation to escape self-awareness. To date, however, the role of self-conscious emotions (shame and guilt) in triggering escape responses after failure has not been sufficiently addressed. In this pre-registered study (N = 156 undergraduates), we adapted a classic paradigm (avoidance of one’s image in a mirror) to a modern eye-tracking technology to test the hypothesis that shame proneness moderates the effect of failure on self-awareness avoidance. Individual differences in guilt and shame proneness were assessed before priming (...) thoughts of failure or success. Then, an eye-tracking paradigm was used to monitor gaze avoidance of one’s screen-reflected face during a neutral, unrelated task. Unexpectedly, results showed that guilt but not shame proneness exacerbated self-avoidance after failure. The present findings challenge the dominant view that shame fosters avoidance more so than guilt. (shrink)

The visual behaviour of consumers buying products in a supermarket was measured and used to analyse the stages of their decision process. Traditionally metrics used to trace decision-making processes are difficult to use in natural environments that often contain many options and unstructured information. Unlike previous attempts in this direction, our methodology reveals differences between a decision-making task and a search task. In particular the second stage of a decision task contains more re-dwells than the second stage of a comparable (...) search task. This study addresses the growing concern of taking eye movement research from the laboratory into the ‘real-world’, so findings can be better generalised to natural situations. (shrink)

Several recent studies have demonstrated context-based, high-confidence misperceptions in hearing, referred to as false hearing. These studies have unanimously found that older adults are more susceptible to false hearing than are younger adults, which the authors have attributed to an age-related decline in the ability to inhibit the activation of a contextually predicted response. However, no published work has investigated this activation-based account of false hearing. In the present study, younger and older adults listened to sentences in which the semantic (...) context provided by the sentence was either unpredictive, highly predictive and valid, or highly predictive and misleading with relation to a sentence-final word in noise. Participants were tasked with clicking on one of four images to indicate which image depicted the sentence-final word in noise. We used eye-tracking to investigate how activation, as revealed in patterns of fixations, of different response options changed in real-time over the course of sentences. We found that both younger and older adults exhibited anticipatory activation of the target word when highly predictive contextual cues were available. When these contextual cues were misleading, younger adults were able to suppress the activation of the contextually predicted word to a greater extent than older adults. These findings are interpreted as evidence for an activation-based model of speech perception and for the role of inhibitory control in false hearing. (shrink)

Multimodal Critical Discourse Analysis (MCDA) as a sub-discipline of Critical Discourse Analysis (CDA) emerged from the availability of social semiotic frameworks describing multimodal meaning making. However, weaknesses of these frameworks have raised concerns and prompted recent methodological reflections in MCDA. Inspired by these reflections, this paper critically assesses MCDA research on advertising persuasion and identifies a lack of attention in studies to account for the social and ideological impact of advertising. This shortcoming is argued to be attributable to the weak (...) empirical grounding of social semiotic frameworks which in turn is rooted in a ‘discursive gap’ (Bateman, Citation2019, p. 300) in multimodality as a discipline. The authors then propose methodological solutions involving eye tracking experiments and higher order data analytics which can strengthen the empirical basis of theoretical assumptions made in MCDA research on advertising. Drawing on methodological guidance for empirical multimodal research, the authors discuss specific issues in realizing this proposal, focusing on a strategy addressing operationalization challenges in experimental design and the affordance of a data cluster algorithm that calculates the common viewing path of a group of viewers. (shrink)

How do people evaluate causal relationships? Do they just consider what actually happened, or do they also consider what could have counterfactually happened? Using eye tracking and Gaussian process modeling, we investigated how people mentally simulated past events to judge what caused the outcomes to occur. Participants played a virtual ball‐shooting game and then—while looking at a blank screen—mentally simulated (a) what actually happened, (b) what counterfactually could have happened, or (c) what caused the outcome to happen. Our findings (...) showed that participants moved their eyes in patterns consistent with the actual or counterfactual events that they mentally simulated. When simulating what caused the outcome to occur, participants moved their eyes consistent with simulations of counterfactual possibilities. These results favor counterfactual theories of causal reasoning, demonstrate how eye movements can reflect simulation during this reasoning and provide a novel approach for investigating retrospective causal reasoning and counterfactual thinking. (shrink)

Speech disfluencies can signal that a speaker is about to refer to something difficult to name. In two experiments, we found evidence that 4-year-olds, like adults, flexibly interpret a particular partner’s disfluency based on their estimate of that partner’s knowledge, derived from the preceding conversation. In entrainment trials, children established partner-specific shared knowledge of names for tangram pictures with one or two adult interlocutors. In each test trial, an adult named one of two visible tangrams either fluently or disfluently while (...) children’s eye-movements were monitored. We manipulated speaker knowledge in the test trials. In Experiment 1, the test-trial speaker was the same speaker from entrainment or a naïve experimenter; in Experiment 2, the test-trial speaker had been one of the child’s partners in entrainment and had seen half of the tangrams. When hearing disfluent expressions, children looked more at a tangram that was unfamiliar from the speaker’s perspective; this systematic disfluency effect disappeared in Experiment 1 when the speaker was entirely naïve, and depended on each speaker’s entrainment experience in Experiment 2. These findings show that 4-year-olds can keep track of two different partners’ knowledge states, and use this information to determine what should be difficult for a particular partner to name, doing so efficiently enough to guide online interpretation of disfluent speech. (shrink)

Access to communication is critical for individuals with late-stage amyotrophic lateral sclerosis and minimal volitional movement, but they sometimes present with concomitant visual or ocular motility impairments that affect their performance with eye tracking or visual brain-computer interface systems. In this study, we explored the use of modified eye tracking and steady state visual evoked potential BCI, in combination with the Shuffle Speller typing interface, for this population. Two participants with late-stage ALS, visual impairments, and minimal volitional movement (...) completed a single-case experimental research design comparing copy-spelling performance with three different typing systems: commercially available eye tracking communication software, Shuffle Speller with modified eye tracking, and Shuffle Speller with SSVEP BCI. Participant 1 was unable to type any correct characters with the commercial system, but achieved accuracies of up to 50% with Shuffle Speller eye tracking and 89% with Shuffle Speller BCI. Participant 2 also had higher maximum accuracies with Shuffle Speller, typing with up to 63% accuracy with eye tracking and 100% accuracy with BCI. However, participants’ typing accuracy for both Shuffle Speller conditions was highly variable, particularly in the BCI condition. Both the Shuffle Speller interface and SSVEP BCI input show promise for improving typing performance for people with late-stage ALS. Further development of innovative BCI systems for this population is needed. (shrink)

Previous processing studies have shown that constituents that are prosodically marked as focus lead to an activation of alternatives. We investigate the processing of constituents that are prosodically marked as contrastive topics. In German, contrastive topics are prosodically realized by prenuclear L*+H accents. Our study tests a) whether prenuclear accents (as opposed to nuclear accents) are able to activate contrastive alternatives, b) whether they do this in the same way as constituents prosodically marked as focus with nuclear accents do, which (...) is important for semantic modeling and c) whether the activation of alternatives is caused by pitch accent type (prenuclear L*+H as contrastive accent vs. prenuclear L+H* as non-contrastive accent) or by differences in F0-excursion (related to prominence). We conducted two visual-world eye-tracking studies, in which German listeners heard declarative utterances (e.g., The swimmer wanted to put on flappers) and watched displays that depicted four printed words: one that was a contrastive alternative to the subject noun (e.g., diver), one that was non-contrastively related to it (e.g., sports), the object (e.g., flappers), which had to be clicked, and an unrelated distractor. Experiment 1 presented participants with two naturally produced intonation conditions, a broad focus control condition with a prenuclear L+H* accent on the subject and a contrastive topic condition with a prenuclear L*+H accent. The results showed that participants fixated more on the contrastive alternative when the subject was produced with an L*+H accent, with the same effect size and timing as reported for focus constituents. Experiment 2 resynthesized the stimuli so that peak height and F0-excursion were the same across intonation conditions. The effect was the same, but the time course was slightly later. Our results suggest that prenuclear L*+H immediately leads to the activation of alternatives during online processing, and that the F0-excursion of the accent lends little. The results are discussed with regard to the processing of contrastive focus accents and theories of contrastive topic. (shrink)

In online lectures, unlike in face-to-face lectures, teachers lack access to (nonverbal) cues to check if their students are still “with them” and comprehend the lecture. The increasing availability of low-cost eye-trackers provides a promising solution. These devices measure unobtrusively where students look and can visualize these data to teachers. These visualizations might inform teachers about students’ level of “with-me-ness” (i.e., do students look at the information that the teacher is currently talking about) and comprehension of the lecture, provided (...) that (1) gaze measures of “with-me-ness” are related to comprehension, (2) people not trained in eye-tracking can predict students’ comprehension from gaze visualizations, (3) we understand how different visualization techniques impact this prediction. We addressed these issues in two studies. In Study 1, 36 students watched a video lecture while being eye-tracked. The extent to which students looked at relevant information and the extent to which they looked at the same location as the teacher both correlated with students’ comprehension (score on an open question) of the lecture. In Study 2, 50 participants watched visualizations of students’ gaze (from Study 1), using six visualization techniques (dynamic and static versions of scanpaths, heatmaps, and focus maps) and were asked to predict students’ posttest performance and to rate their ease of prediction. We found that people can use gaze visualizations to predict learners’ comprehension above chance level, with minor differences between visualization techniques. Further research should investigate if teachers can act on the information provided by gaze visualizations and thereby improve students’ learning. (shrink)

It has been four decades since eye-tracking was first used in interpreting studies, and recent years has witnessed a growing interest in the application of this method, which holds great potential for offering a look into the “black box” of interpreting processing. However, little attention has been paid to comprehensively illustrating what has been done, what can be done, and what needs to be done with this method in this discipline. With this in view, this paper sets out to (...) understand contributions of previous studies—key themes discussed, eye-tracking measures used, their limitations and implications, and future directions. To this end, we conduct a review of a total of 26 empirical papers from peer-reviewed journals within a time span of 4 decades ranging from 1981 to 2021. This study, as the first attempt of its kind at a comprehensive review on using eye-tracking in interpreting studies, should have implications for researchers, educators, and practitioners. (shrink)

Two visual world experiments investigated the processing of the implicature associated with some using a “gumball paradigm.” On each trial, participants saw an image of a gumball machine with an upper chamber with orange and blue gumballs and an empty lower chamber. Gumballs dropped to the lower chamber, creating a contrast between a partitioned set of gumballs of one color and an unpartitioned set of the other. Participants then evaluated spoken statements, such as “You got some of the blue gumballs.” (...) Experiment 1 investigated the time course of the pragmatic enrichment from some to not all when the only utterance alternatives available to refer to the different sets were some and all. In Experiment 2, the number terms two, three, four, and five were also included in the set of alternatives. Scalar implicatures were delayed relative to the interpretation of literal statements with all only when number terms were available. The results are interpreted as evidence for a constraint-based account of scalar implicature processing. (shrink)

Cognitive architectures have often been applied to data from individual experiments. In this paper, I develop an ACT-R reader that can model a much larger set of data, eye-tracking corpus data. It is shown that the resulting model has a good fit to the data for the considered low-level processes. Unlike previous related works, the model achieves the fit by estimating free parameters of ACT-R using Bayesian estimation and Markov-Chain Monte Carlo techniques, rather than by relying on the mix (...) of manual selection + default values. The method used in the paper is generalizable beyond this particular model and data set and could be used on other ACT-R models. (shrink)

Background: Facial attractiveness influences our perceptions of others, with beautiful faces reaping societal rewards and anomalous faces encountering penalties. The purpose of this study was to determine associations of visual attention with bias and social dispositions toward people with facial anomalies. -/- Methods: Sixty subjects completed tests evaluating implicit bias, explicit bias, and social dispositions before viewing publicly available images of preoperative and postoperative patients with hemifacial microsomia. Eye-tracking was used to register visual fixations. -/- Results: Participants with higher (...) implicit bias scores fixated significantly less on the cheek and ear region preoperatively (P = 0.004). Participants with higher scores in empathic concern and perspective taking fixated more on the forehead and orbit preoperatively (P = 0.045) and nose and lips (P = 0.027) preoperativel. -/- Conclusions: Participants with higher levels of implicit bias spent less visual attention on anomalous facial anatomy, whereas participants with higher levels of empathic concern and perspective taking spent more visual attention on normal facial anatomy. Levels of bias and social dispositions such as empathy may predict layperson gaze patterns toward those with facial anomalies and provide insights to neural mechanisms underlying the "anomalous is bad" paradigm. (shrink)

When people are asked to estimate the probabilities of uncertain events, they often neglect the additivity principle, which requires that the probabilities assigned to an exhaustive set of outcomes should add up to 100%. Previous studies indicate that additivity neglect is dependent on response format, self-generated probability estimates being more coherent than estimates on rating scales. The present study made use of eye-tracking methodology, recording the movement, frequency and duration of fixations during the solution of ten additivity problems and (...) two control tasks. Participants produced more non-additive estimates in the Scale format than in the Self-generated format. Self-generated estimates also led to longer decision time and a higher number of repeated inspections, suggesting a deliberate comparison process. In contrast, the Scale format seemed to encourage a case-based approach where each outcome is evaluated in isolation. (shrink)

Unilateral spatial neglect is a disorder characterized by the failure to report, respond to, or orient toward the contralateral side of space to a brain lesion. Current assessment methods often fail to discover milder forms, cannot differentiate between unilateral spatial neglect subtypes and lack ecological validity. There is also a need for treatment methods that target subtypes. Immersive virtual reality systems in combination with eye-tracking have the potential to overcome these shortcomings, by providing more naturalistic environments and tasks, with (...) sensitive and detailed measures. This systematic review examines the state of the art of research on these technologies as applied in the assessment and treatment of USN. As we found no studies that combined immersive VR and ET, we reviewed these approaches individually. The review of VR included seven articles, the ET review twelve. The reviews revealed promising results. All included studies found significant group-level differences for several USN measures. In addition, several studies found asymmetric behavior in VR and ET tasks for patients who did not show signs of USN in conventional tests. Particularly promising features were multitasking in complex VR environments and detailed eye-movement analysis. No VR and only a few ET studies attempted to differentiate USN subtypes, although the technologies appeared appropriate. One ET study grouped USN participants using individual heatmaps, and another differentiated between subtypes on drawing tasks. Regarding ecological validity, although no studies tested the prognostic validity of their assessment methods, VR and ET studies utilized naturalistic tasks and stimuli reflecting everyday situations. Technological characteristics, such as the field of view and refresh rate of the head-mounted displays, could be improved, though, to improve ecological validity. We found no studies that utilized VR or ET technologies for USN treatment up until the search date of the 26th of February 2020. In conclusion, VR-ET-based systems show great potential for USN assessment. VR-ET holds great promise for treatment, for example, by monitoring behavior and adapting and tailoring to the individual person’s needs and abilities. Future research should consider developing methods for individual subtypes and differential diagnostics to inform individual treatment programs. (shrink)

Prior research has shown that various types of conventional multiword chunks are processed faster than matched novel strings, but it is unclear whether this processing advantage extends to variant multiword chunks that are less formulaic. To determine whether the processing advantage of multiword chunks accommodates variations in the canonical phrasal template, we examined the robustness of the processing advantage (i.e., predictability) of binomial phrases with non‐canonical conjunctions (e.g.,salt and also pepper; salt as well as pepper). Results from the cloze study (...) (Experiment 1) showed that there was a high tendency of producing the canonical conjunct (pepper), even in the binomials that contained non‐formulaic conjunctions. Consistent with these findings, results from two eye tracking studies (Experiments 2a and 2b) showed that canonical conjuncts were read faster than novel conjuncts that were matched on word length (e.g.,paprika), even in the binomials with variant conjunctions. This robust online processing advantage was replicated in a self‐paced reading study that compared all three conjunction types (Experiment 3). Taken together, these findings show that binomials with variant function words also receive facilitated processing relative to matched novel strings, even though both types of strings are neither conventional nor relatively frequent. Exploratory analyses revealed that this processing speed advantage was driven by the lexical–semantic association between the canonical conjuncts (salt–pepper), rather than lexical and phrasal frequency. Overall, these results highlight flexibility in the processing of multiword chunks that current models of multiword storage and processing must take into account. (shrink)