It has been four decades since eye-tracking was first used in interpreting studies, and recent years has witnessed a growing interest in the application of this method, which holds great potential for offering a look into the “black box” of interpreting processing. However, little attention has been paid to comprehensively illustrating what has been done, what can be done, and what needs to be done with this method in this discipline. With this in view, this paper sets out to understand (...) contributions of previous studies—key themes discussed, eye-tracking measures used, their limitations and implications, and future directions. To this end, we conduct a review of a total of 26 empirical papers from peer-reviewed journals within a time span of 4 decades ranging from 1981 to 2021. This study, as the first attempt of its kind at a comprehensive review on using eye-tracking in interpreting studies, should have implications for researchers, educators, and practitioners. (shrink)

All other things being equal the parser favors attaching an ambiguous modifier to the most recent possible site. A plausible explanation is that locality preferences such as this arise in the service of minimizing memory costs—more distant sentential material is more difficult to reactivate than more recent material. Note that processing any sentence requires linking each new lexical item with material in the current parse. This often involves the construction of long‐distance dependencies. Under a resource‐limited view of language processing, lengthy (...) integrations should induce difficulty even in unambiguous sentences. To date there has been little direct quantitative evidence in support of this perspective. This article presents 2 self‐paced reading studies, which explore the hypothesis that dependency distance is a fundamental determinant of reading complexity in unambiguous constructions in English. The evidence suggests that the difficulty associated with integrating a new input item is heavily determined by the amount of lexical material intervening between the input item and the site of its target dependents. The patterns observed here are not straightforwardly accounted for within purely experience‐based models of complexity. Instead, this work supports the role of a memory bottleneck in language comprehension. This constraint arises because hierarchical linguistic relations must be recovered from a linear input stream. (shrink)

Two self-paced reading experiments investigated several factors that influence the comprehension complexity of singly-embedded relative clauses (RCs) in English. Three factors were manipulated in Experiment 1, resulting in three main effects. First, object-extracted RCs were read more slowly than subject-extracted RCs, replicating previous work. Second, RCs that were embedded within the sentential complement of a noun were read more slowly than comparable RCs that were not embedded in this way. Third, and most interestingly, object-modifying RCs were read more slowly than (...) subject-modifying relative clauses. This result contradicts one of the central tenets of complexity research: that nested sentences are harder to understand than their right-branching equivalents (e.g., Miller and Chomsky 1963). It is hypothesized that this result followed from a combination of two information-flow factors: (1) background information is usually presented early in a sentence; and (2) restrictive RCs—the form of the RCs in Experiment 1—usually convey background information. Experiment 2 tested this hypothesis by comparing restrictive and non-restrictive RCs— which generally provide new information—in both subject- and objectmodifying positions. The results of the experiment were as predicted by the information-flow account: Only restrictive RCs were read more slowly when modifying objects. It is concluded that both resource and information-flow factors need to be considered in explaining RC complexity effects. (shrink)

Word order alternation has been described as one of the most productive information structure markers and discourse organizers across languages. Psycholinguistic evidence has shown that word order is a crucial cue for argument interpretation. Previous studies about Spanish sentence comprehension have shown greater difficulty to parse sentences that present a word order that does not respect the order of participants of the verb's lexico-semantic structure, irrespective to whether the sentences follow the canonical word order of the language or not. This (...) difficulty has been accounted as the cognitive cost related to the miscomputation of prominence status of the argument that precedes the verb. Nonetheless, the authors only analyzed the use of alternative word orders in isolated sentences, leaving aside the pragmatic motivation of word order alternation. By means of an eye-tracking task, the current study provides further evidence about the role of information structure for the comprehension of sentences with alternative word order and verb type, and sheds light on the interaction between syntax, semantics and pragmatics. We analyzed both “early” and “late” eye-movement measures as well as accuracy and response times to comprehension questions. Results showed an overall influence of information structure reflected in a modulation of late eye-movement measures as well as offline measures like total reading time and questions response time. However, effects related to the miscomputation of prominence status did not fade away when sentences were preceded by a context that led to non-canonical word order of constituents, showing that prominence computation is a core mechanism for argument interpretation, even in sentences preceded by context. (shrink)

We explore the interaction between oculomotor control and language comprehension on the sentence level using two well-tested computational accounts of parsing difficulty. Previous work (Boston, Hale, Vasishth, & Kliegl, 2011) has shown that surprisal (Hale, 2001; Levy, 2008) and cue-based memory retrieval (Lewis & Vasishth, 2005) are significant and complementary predictors of reading time in an eyetracking corpus. It remains an open question how the sentence processor interacts with oculomotor control. Using a simple linking hypothesis proposed in Reichle, Warren, and (...) McConnell (2009), we integrated both measures with the eye movement model EMMA (Salvucci, 2001) inside the cognitive architecture ACT-R (Anderson et al., 2004). We built a reading model that could initiate short “Time Out regressions” (Mitchell, Shen, Green, & Hodgson, 2008) that compensate for slow postlexical processing. This simple interaction enabled the model to predict the re-reading of words based on parsing difficulty. The model was evaluated in different configurations on the prediction of frequency effects on the Potsdam Sentence Corpus. The extension of EMMA with postlexical processing improved its predictions and reproduced re-reading rates and durations with a reasonable fit to the data. This demonstration, based on simple and independently motivated assumptions, serves as a foundational step toward a precise investigation of the interaction between high-level language processing and eye movement control. (shrink)

Domain experts regularly teach novice students how to perform a task. This often requires them to adjust their behavior to the less knowledgeable audience and, hence, to behave in a more didactic manner. Eye movement modeling examples (EMMEs) are a contemporary educational tool for displaying experts’ (natural or didactic) problem‐solving behavior as well as their eye movements to learners. While research on expert‐novice communication mainly focused on experts’ changes in explicit, verbal communication behavior, it is as yet unclear whether and (...) how exactly experts adjust their nonverbal behavior. This study first investigated whether and how experts change their eye movements and mouse clicks (that are displayed in EMMEs) when they perform a task naturally versus teach a task didactically. Programming experts and novices initially debugged short computer codes in a natural manner. We first characterized experts’ natural problem‐solving behavior by contrasting it with that of novices. Then, we explored the changes in experts’ behavior when being subsequently instructed to model their task solution didactically. Experts became more similar to novices on measures associated with experts’ automatized processes (i.e., shorter fixation durations, fewer transitions between code and output per click on the run button when behaving didactically). This adaptation might make it easier for novices to follow or imitate the expert behavior. In contrast, experts became less similar to novices for measures associated with more strategic behavior (i.e., code reading linearity, clicks on run button) when behaving didactically. (shrink)

Domain experts regularly teach novice students how to perform a task. This often requires them to adjust their behavior to the less knowledgeable audience and, hence, to behave in a more didactic manner. Eye movement modeling examples (EMMEs) are a contemporary educational tool for displaying experts’ (natural or didactic) problem‐solving behavior as well as their eye movements to learners. While research on expert‐novice communication mainly focused on experts’ changes in explicit, verbal communication behavior, it is as yet unclear whether and (...) how exactly experts adjust their nonverbal behavior. This study first investigated whether and how experts change their eye movements and mouse clicks (that are displayed in EMMEs) when they perform a task naturally versus teach a task didactically. Programming experts and novices initially debugged short computer codes in a natural manner. We first characterized experts’ natural problem‐solving behavior by contrasting it with that of novices. Then, we explored the changes in experts’ behavior when being subsequently instructed to model their task solution didactically. Experts became more similar to novices on measures associated with experts’ automatized processes (i.e., shorter fixation durations, fewer transitions between code and output per click on the run button when behaving didactically). This adaptation might make it easier for novices to follow or imitate the expert behavior. In contrast, experts became less similar to novices for measures associated with more strategic behavior (i.e., code reading linearity, clicks on run button) when behaving didactically. (shrink)

Remote interpreting via video-link is increasingly being employed in investigative interviews chiefly due to its apparent increased accessibility and efficiency. However, risks of miscommunication have been shown to be magnified in remote interpreting and empirical research specifically on video-link remote interpreting is in its infancy which greatly limits the evidence base available to inform and direct evidence-based policy and best practice, particularly in the identification of the optimal mode of interpreting to be used, namely consecutive and simultaneous. Consecutive interpreting refers (...) to a process in which the interpreter transfers short segments of speech from one language into the other as each person speaks in managed turn-taking, while simultaneous interpreting refers to the transfer of natural speech from one language into another in a concurrent manner without the need for speakers to segment their speech. This study provides novel empirical evidence by using eye tracking to compare the overt visual attention of interpreters working in a remote setting in which an English-speaking Interviewer interacts with a non-English-speaking Suspect in person, for whom interpretation is provided via video-link in real time. Using a within-subject design, we analyze eye-movement data from 28 professionally accredited interpreters who interpreted via video-link an investigative interview in which consecutive and simultaneous interpreting modes were counterbalanced. Taking interpreting performance into account, our results showed that, the consecutive mode yielded significantly less gaze time and therefore significantly less on-screen overt visual attention due to off-screen notetaking, an essential component of the consecutive interpreting mode. Relative to gaze time, the consecutive mode also resulted in significantly more and longer fixations and shifts of attention. Participants also allocated significantly more overt visual attention to the Interviewer than the Suspect, particularly in the consecutive mode. Furthermore, we found informative significant correlations between eye tracking measures and interpreting performance: accuracy, verbal rapport, and management. Finally, we found no significant differences between the three language pairs tested. We conclude with a discussion of limitations and the contributions of the study and an outline for future work on this topic of growing importance. (shrink)

Object relative clauses are harder to process than subject relative clauses. Under Grillo’s Generalized Minimality framework, complexity effects of object relatives are construed as intervention effects, which result from an interaction between locality constraints on movement and the sentence processing system. Specifically, intervention of the subject DP in the movement dependency is expected to generate a minimality violation whenever processing limitations render the moved object underspecified, resulting in compromised comprehension. In the present study, assuming Generalized Minimality, we compared the processing (...) of object relatives with the processing of subject control in ditransitives, which, like object relatives, instantiates a syntactic dependency across an intervening DP. This comparison is justified by the current debate on whether Control should be analyzed as movement: if control involves movement of the controller DP, as proposed by Hornstein, a parallel between the processing of object relatives and subject control in ditransitives may be anticipated on the basis of intervention. In addition, we explored whether general cognitive factors contribute to complexity effects ascribed to movement across a DP. Sixty-nine adult speakers of European Portuguese read sentences and answered comprehension probes in a self-paced reading task with moving-window display, comprising four experimental conditions: Subject Relatives; Object Relatives; Subject Control; Object Control. Furthermore, participants performed four supplementary tasks, serving as measures of resistance to interference, lexical knowledge, working memory capacity and lexical access ability. The results from the reading task showed that whereas object relatives were harder to process than subject relatives, subject control was not harder to process than object control, arguing against recent movement accounts of control. Furthermore, we found that whereas object relative complexity effects assessed by response times to comprehension probes interacted with Reading Span, object relative complexity effects assessed by comprehension accuracy and reading times did not interact with any of the supplementary tasks. We discuss these results in light of Generalized Minimality and the hypothesis of modularity in syntactic processing. (shrink)

Both linguistic and extralinguistic consultations are essential in translation practice and have been commonly investigated as an integral topic in previous studies. However, since extralinguistic information is usually longer in extent and not specifically designed for a linguistic purpose, extralinguistic consultations involve different search strategies compared with linguistic consultations. Drawing on eye-tracking and screen-recording data, this study compares linguistic and extralinguistic consultations in terms of cognitive resources allocation and information processing patterns in English–Chinese translation. It also explores the differences among (...) 17 language learners, 20 student translators, and 21 professional translators, and the effect of extralinguistic consultation on their translation quality. The findings are as follows: all participants allocate more attention and lower cognitive load to extralinguistic consultations than to linguistic consultations; participants’ translation experience levels and their attention allocated to extralinguistic consultation show an inverted U-shaped relationship; and participants who consult extralinguistic information before drafting or devote more attention to extralinguistic consultation produce target texts with significantly higher scores. (shrink)

Social robots serve people as tutors, caretakers, receptionists, companions, and other social agents. People know that the robots are mechanical artifacts, yet they interact with them as if they were actual agents. How is this possible? The proposal here is that people construe social robots not as social agents per se, but as depictions of social agents. They interpret them much as they interpret ventriloquist dummies, hand puppets, virtual assistants, and other interactive depictions of people and animals. Depictions as a (...) class consist of three physical scenes with part-by-part mappings between them: (a) a base scene (the raw physical artifact), (b) the depiction proper (the artifact construed as a depiction), and (c) the scene depicted (the scene people are to imagine). With social robots, evidence shows people form the same three scenes plus mappings: They perceive the raw machinery of a robot, construe it as a depiction of a character, and, using the depiction as a guide, engage in the pretense that they are interacting with the character depicted. With social robots, people also recognize three classes of agents – the characters depicted, the intended recipients of the depictions (those who view or interact with the robots), and the authorities responsible for the robots (the designers, makers, and owners). Construing social robots as depictions, we argue, accounts for many phenomena not covered by alternative models. (shrink)

Modern consumers face a dramatic rise in web-based technological advancements and have trouble making rational and proper decisions when they shop online. When they try to make decisions about products and services, they also feel pressured against time when sorting among all of the unnecessary items in the flood of information available on the web. In this sense, they need to use consumer decision-making creativity to make rational decisions. However, unexplored research questions on this subject remain. First, in what ways (...) do task difficulty and time constraints affect visual attention on exploitative and exploratory activities differently? Second, how does the location of the reference influence the level of visual attention to exploitative and exploratory activities depending on affordance theory? Third, how do exploratory and exploitative activities affect CDMC? Eye-tracking experiments were conducted with 70 participants to obtain relevant metrics such as total fixation duration, fixation count, and visit count to answer these research questions. Our findings suggest that task difficulty influences exploitative activity, whereas time constraint is related to the exploratory activity. The result of the location of hints aligns with the affordance theory for the exploitative activity. Besides, exploratory activity positively affected CDMC, but exploitative activity did not show any effect. (shrink)

This study investigated the impact of online product reviews on consumers purchasing decisions by using eye-tracking. The research methodology involved development of a conceptual framework of online product review and purchasing intention through the moderation role of gender and visual attention in comments, and empirical investigation into the region of interest analysis of consumers fixation during the purchase decision process and behavioral analysis. The results showed that consumers’ attention to negative comments was significantly greater than that to positive comments, especially (...) for female consumers. Furthermore, the study identified a significant correlation between the visual browsing behavior of consumers and their purchase intention. It also found that consumers were not able to identify false comments. The current study provides a deep understanding of the underlying mechanism of how online reviews influence shopping behavior, reveals the effect of gender on this effect for the first time and explains it from the perspective of attentional bias, which is essential for the theory of online consumer behavior. Specifically, the different effects of consumers’ attention to negative comments seem to be moderated through gender with female consumers’ attention to negative comments being significantly greater than to positive ones. These findings suggest that practitioners need to pay particular attention to negative comments and resolve them promptly through the customization of product/service information, taking into consideration consumer characteristics, including gender. (shrink)

An eye-tracking while listening study based on the blank screen paradigm was conducted to investigate the processing of literal and metaphorical verbs of motion. The study was based on two assumpti...

We present interpretation‐based processing—a theory of sentence processing that builds a syntactic and a semantic representation for a sentence and assigns an interpretation to the sentence as soon as possible. That interpretation can further participate in comprehension and in lexical processing and is vital for relating the sentence to the prior discourse. Our theory offers a unified account of the processing of literal sentences, metaphoric sentences, and sentences containing semantic illusions. It also explains how text can prime lexical access. We (...) show that word literality is a matter of degree and that the speed and quality of comprehension depend both on how similar words are to their antecedents in the preceding text and how salient the sentence is with respect to the preceding text. Interpretation‐based processing also reconciles superficially contradictory findings about the difference in processing times for metaphors and literals. The theory has been implemented in ACT‐R [Anderson and Lebiere, The Atomic Components of Thought, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, 1998]. (shrink)