Eye movements during mental imagery are not epiphenomenal but assist the process of image generation. Commands to the eyes for each fixation are stored along with the visual representation and are used as spatial index in a motor‐based coordinate system for the proper arrangement of parts of an image. In two experiments, subjects viewed an irregular checkerboard or color pictures of fish and were subsequently asked to form mental images of these stimuli while keeping their eyes open. During the perceptual (...) phase, a group of subjects was requested to maintain fixation onto the screen's center, whereas another group was free to inspect the stimuli. During the imagery phase, all of these subjects were free to move their eyes. A third group of subjects (in Experiment 2) was free to explore the pattern but was requested to maintain central fixation during imagery. For subjects free to explore the pattern, the percentage of time spent fixating a specific location during perception was highly correlated with the time spent on the same (empty) locations during imagery. The order of scanning of these locations during imagery was correlated to the original order during perception. The strength of relatedness of these scanpaths and the vividness of each image predicted performance accuracy. Subjects who fixed their gaze centrally during perception did the same spontaneously during imagery. Subjects free to explore during perception, but maintaining central fixation during imagery, showed decreased ability to recall the pattern. We conclude that the eye scanpaths during visual imagery reenact those of perception of the same visual scene and that they play a functional role. (shrink)

Scanpaths have played an important role in classic research on reading behavior. Nevertheless, they have largely been neglected in later research perhaps due to a lack of suitable analytical tools. Recently, von der Malsburg and Vasishth proposed a new measure for quantifying differences between scanpaths and demonstrated that this measure can recover effects that were missed with the traditional eyetracking measures. However, the sentences used in that study were difficult to process and scanpath effects accordingly strong. The purpose (...) of the present study was to test the validity, sensitivity, and scope of applicability of the scanpath measure, using simple sentences that are typically read from left to right. We derived predictions for the regularity of scanpaths from the literature on oculomotor control, sentence processing, and cognitive aging and tested these predictions using the scanpath measure and a large database of eye movements. All predictions were confirmed: Sentences with short words and syntactically more difficult sentences elicited more irregular scanpaths. Also, older readers produced more irregular scanpaths than younger readers. In addition, we found an effect that was not reported earlier: Syntax had a smaller influence on the eye movements of older readers than on those of young readers. We discuss this interaction of syntactic parsing cost with age in terms of shifts in processing strategies and a decline of executive control as readers age. Overall, our results demonstrate the validity and sensitivity of the scanpath measure and thus establish it as a productive and versatile tool for reading research. (shrink)

In the last decade, we have observed a continuous increase of interest in eye movement research. According to a recent investigation, eye movements are discussed in over one million publications. The number of publications with eye movement in the title or abstract has been steadily increasing over the years, with over 1,200 papers published alone in 2013. The last decade has also witnessed the emergence of many new sub-disciplines in the field of neuroscience and cognition - one of them is (...) neuroaesthetics, which refers to the [neuro-]science of aesthetic perception of art.The title and contents of our book have been inspired by a very influential research article. It is one of the most cited scientific papers of all time, published in 1959 by a team of neurophysiologists and engineers, Lettvin, McCulloch and Pitts who are considered to be the founders of modern cybernetics. Their article, "What the Frog's Eye Tells the Frog's Brain," refers to the role of internal cortical models in the communication or interfacing of the information in the outside world with the practical contextual task of the viewer. It shows how eye movements are the modality of this communication. The same duality between eye movements and internal models plays a fundamental role in humans. When we look, for example, at art, it explains those neurological processes involved in neuroaesthetics. Our book undertakes this innovative approach to neuroaesthetics. It explains this duality and discusses the communication between the artist and the viewer's aesthetic perception. It is structured into five chapters.Chapter One discusses the neurology of aesthetics, the idea of art as a form of communication, and explains perception as an active matching between a "top" (the viewer's mind) and a "down" (the viewer's sensorial machinery). It goes deep into the philosophical quandary of what beauty is in terms of art. Chapter Two is about eye movements and the scanpath theory of vision perception. It discusses the role of visual attention for controlling active vision, the meaning of mental binding and the analysis of eye movements as the key to understanding aesthetic processes. Chapter Three introduces Claude Shannon's information theory using it as a matrix into which to embed the top down active vision scanpath theory. The authors explain the main concept of neuroaesthetics as a form of communication mediated by the sequence of eye fixations. Chapter Four discusses art critique: The role of the viewer's training and expectation, the dilemma of the aesthetics of art versus non-art and how all of this affects the viewing mode. Finally, Chapter Five treats the intimacy of the artistic process, showing the unique implementation of the communicative experience between the artist and the viewer - the relation between a pictorial representation defined by the artist and modes of AWE generated in the viewer during active looking. (shrink)

Although eye tracking has been used extensively to assess cognitions for static stimuli, recent research suggests that the link between gaze and cognition may be more tenuous for dynamic stimuli such as videos. Part of the difficulty in convincingly linking gaze with cognition is that in dynamic stimuli, gaze position is strongly influenced by exogenous cues such as object motion. However, tests of the gaze‐cognition link in dynamic stimuli have been done on only a limited range of stimuli often characterized (...) by highly organized motion. Also, analyses of cognitive contrasts between participants have been mostly been limited to categorical contrasts among small numbers of participants that may have limited the power to observe more subtle influences. We, therefore, tested for cognitive influences on gaze for screen‐captured instructional videos, the contents of which participants were tested on. Between‐participant scanpath similarity predicted between‐participant similarity in responses on test questions, but with imperfect consistency across videos. We also observed that basic gaze parameters and measures of attention to centers of interest only inconsistently predicted learning, and that correlations between gaze and centers of interest defined by other‐participant gaze and cursor movement did not predict learning. It, therefore, appears that the search for eye movement indices of cognition during dynamic naturalistic stimuli may be fruitful, but we also agree that the tyranny of dynamic stimuli is real, and that links between eye movements and cognition are highly dependent on task and stimulus properties. (shrink)

In online lectures, unlike in face-to-face lectures, teachers lack access to (nonverbal) cues to check if their students are still “with them” and comprehend the lecture. The increasing availability of low-cost eye-trackers provides a promising solution. These devices measure unobtrusively where students look and can visualize these data to teachers. These visualizations might inform teachers about students’ level of “with-me-ness” (i.e., do students look at the information that the teacher is currently talking about) and comprehension of the lecture, provided that (...) (1) gaze measures of “with-me-ness” are related to comprehension, (2) people not trained in eye-tracking can predict students’ comprehension from gaze visualizations, (3) we understand how different visualization techniques impact this prediction. We addressed these issues in two studies. In Study 1, 36 students watched a video lecture while being eye-tracked. The extent to which students looked at relevant information and the extent to which they looked at the same location as the teacher both correlated with students’ comprehension (score on an open question) of the lecture. In Study 2, 50 participants watched visualizations of students’ gaze (from Study 1), using six visualization techniques (dynamic and static versions of scanpaths, heatmaps, and focus maps) and were asked to predict students’ posttest performance and to rate their ease of prediction. We found that people can use gaze visualizations to predict learners’ comprehension above chance level, with minor differences between visualization techniques. Further research should investigate if teachers can act on the information provided by gaze visualizations and thereby improve students’ learning. (shrink)