When we read narrative texts such as novels and newspaper articles, we segment information presented in such texts into discrete events, with distinct boundaries between those events. But do our eyes reflect this event structure while reading? This study examines whether eye movements during the reading of discourse reveal how readers respond online to event structure. Participants read narrative passages as we monitored their eye movements. Several measures revealed that event structure predicted eye movements. In two experiments, we found that (...) both early and overall reading times were longer for event boundaries. We also found that regressive saccades were more likely to land on event boundaries, but that readers were less likely to regress out of an event boundary. Experiment 2 also demonstrated that tracking event structure carries a working memory load. Eye movements provide a rich set of online data to test the cognitive reality of event segmentation during reading. (shrink)

Shin and DuBrow propose that a key principle driving event segmentation relates to causal analyses: specifically, that experiences that are attributed as having the same underlying cause are grouped together into an event. This offers an alternative to accounts of segmentation based on prediction error.

Summary This article questions certain assumptions concerning film form made by the recent psychological film research and compares them to those of precursors of film psychology like Hugo Münsterberg and Rudolf Arnheim, as well as the principles of Gestalt psychology. It is argued that principles of Gestalt psychology such as those of ‘good form’ and good continuation are still underlying the psychological research of film, becoming particularly apparent in its approach to continuity editing. Following an alternative Gestalt genealogy that links (...) Gestalt theory with more recent dynamic models of brain activity and with accounts of brain complexity and neuronal synchronisation, the article concludes that psychological research on film needs to shift the focus from form to transformation, both in conceiving the perceptual and cognitive processing of films and in approaching film aesthetics more broadly. (shrink)

A fundamental aspect of human cognition is the ability to parse our constantly unfolding experience into meaningful representations of dynamic events and to communicate about these events with others. How do we communicate about events we have experienced? Influential theories of language production assume that the formulation and articulation of a linguistic message is preceded by preverbal apprehension that captures core aspects of the event. Yet the nature of these preverbal event representations and the way they are mapped onto language (...) are currently not well understood. Here, we review recent evidence on the link between event conceptualization and language, focusing on two core aspects of event representation: event roles and event boundaries. Empirical evidence in both domains shows that the cognitive representation of events aligns with the way these aspects of events are encoded in language, providing support for the presence of deep homologies between linguistic and cognitive event structure. (shrink)

Filmmakers use continuity editing to engender a sense of situational continuity or discontinuity at editing boundaries. The goal of this study was to assess the impact of continuity editing on how people perceive the structure of events in a narrative film and to identify brain networks that are associated with the processing of different types of continuity editing boundaries. Participants viewed a commercially produced film and segmented it into meaningful events, while brain activity was recorded with functional magnetic resonance imaging (...) (MRI). We identified three degrees of continuity that can occur at editing locations: edits that are continuous in space, time, and action; edits that are discontinuous in space or time but continuous in action; and edits that are discontinuous in action as well as space or time. Discontinuities in action had the biggest impact on behavioral event segmentation, and discontinuities in space and time had minor effects. Edits were associated with large transient increases in early visual areas. Spatial-temporal changes and action changes produced strikingly different patterns of transient change, and they provided evidence that specialized mechanisms in higher order perceptual processing regions are engaged to maintain continuity of action in the face of spatiotemporal discontinuities. These results suggest that commercial film editing is shaped to support the comprehension of meaningful events that bridge breaks in low-level visual continuity, and even breaks in continuity of spatial and temporal location. (shrink)

Understanding how people comprehend visual narratives (including picture stories, comics, and film) requires the combination of traditionally separate theories that span the initial sensory and perceptual processing of complex visual scenes, the perception of events over time, and comprehension of narratives. Existing piecemeal approaches fail to capture the interplay between these levels of processing. Here, we propose the Scene Perception & Event Comprehension Theory (SPECT), as applied to visual narratives, which distinguishes between front-end and back-end cognitive processes. Front-end processes occur (...) during single eye fixations and are comprised of attentional selection and information extraction. Back-end processes occur across multiple fixations and support the construction of event models, which reflect understanding of what is happening now in a narrative (stored in working memory) and over the course of the entire narrative (stored in long-term episodic memory). We describe relationships between front- and back-end processes, and medium-specific differences that likely produce variation in front-end and back-end processes across media (e.g., picture stories vs. film). We describe several novel research questions derived from SPECT that we have explored. By addressing these questions, we provide greater insight into how attention, information extraction, and event model processes are dynamically coordinated to perceive and understand complex naturalistic visual events in narratives and the real world. (shrink)

Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT‐2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT‐2 to compute the time series of (...) prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT‐2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of prediction error. (shrink)

Because visual narratives demand complex inference abilities, they can potentially be used as a tool for developing inferential skills in other domains, like reading. The Inferential Language Comprehension (iLC) Framework proposes an approach to using visual narratives in educational settings to sponsor inference skills by building on cognitive, developmental, and language research.

Reading fiction for pleasurable is robustly correlated with improved cognitive attainment and other benefits. It is also in decline among young people in developed nations, in part because of competition from moving image fiction. We review existing research on the differences between reading/hearing verbal fiction and watching moving image fiction, as well as looking more broadly at research on image/text interactions and visual versus verbal processing. We conclude that verbal narrative generates more diverse responses than moving image narrative., We note (...) that reading and viewing narrative are different tasks, with different cognitive loads. Viewing moving image narrative mostly involves visual processing with some working memory engagement, whereas reading narrative involves verbal processing, visual imagery and personal memory (Xu et al 2005). but Attempts to compare the two by creating suggest that existing research is flawed by attempts to create equivalent stimuli and task demands face a number of challenges, and we discuss these difficulties in comparative approaches. We then investigate the possibility of identifying lower level processing mechanisms that might distinguish cognition of the two media, and propose a focus on internal scene construction and on working memory as foci for future research. Although many of the sources we draw on are focussed on English speaking participants in European or North American settings, we also cover material relating to speakers of Dutch, German, Hebrew and Japanese in their respective countries, and studies of a remote Turkish mountain community. (shrink)

Verbal narratives provide incomplete information and can be very long, yet readers and hearers often effortlessly fill in the gaps and make connections across long stretches of text, sometimes even finding this immersive. How is this done? In the last few decades, event-indexing situation modeling and complementary accounts of narrative emotion have suggested answers. Despite this progress, comparisons between real-life perception and narrative experience might underplay the way narrative processing modifies our world model, as well as the role of the (...) emotions that do not relate to characters. I reframe narrative experience in predictive processing and neural networks, capturing continuity between fiction, perception, and states like dreaming and imagination, enabled by the flexible instantiation of concepts. In this framework, narrative experience is more clearly revealed as a creative experience that can share some of the phenomenology of dreams. (shrink)

What role do moment‐to‐moment comprehension processes play in visual attentional selection in picture stories? The current work uniquely tested the role of bridging inference generation processes on eye movements while participants viewed picture stories. Specific components of the Scene Perception and Event Comprehension Theory (SPECT) were tested. Bridging inference generation was induced by manipulating the presence of highly inferable actions embedded in picture stories. When inferable actions are missing, participants have increased viewing times for the immediately following critical image (Magliano, (...) Larson, Higgs, & Loschky, ). This study used eye‐tracking to test competing hypotheses about the increased viewing time: (a) Computational Load: inference generation processes increase overall computational load, producing longer fixation durations; (b) Visual Search: inference generation processes guide eye‐movements to pick up inference‐relevant information, producing more fixations. Participants had similar fixation durations, but they made more fixations while generating inferences, with that process starting from the fifth fixation. A follow‐up hypothesis predicted that when generating inferences, participants fixate scene regions important for generating the inference. A separate group of participants rated the inferential‐relevance of regions in the critical images, and results showed that these inferentially relevant regions predicted differences in other viewers’ eye movements. Thus, viewers’ event models in working memory affect visual attentional selection while viewing visual narratives. (shrink)

Popular movies present chunk-like events that promote episodic, serial updating of viewers’ representations of the ongoing narrative. Event-indexing theory would suggest that the beginnings of new scenes trigger these updates, which in turn require more cognitive processing. Typically, a new movie event is signaled by an establishing shot, one providing more background information and a longer look than the average shot. Our analysis of 24 films reconfirms this. More important, we show that, when returning to a previously shown location, the (...) re-establishing shot reduces both context and duration while remaining greater than the average shot. In general, location shifts dominate character and time shifts in event segmentation of movies. In addition, over the last 70 years re-establishing shots have become more like the noninitial shots of a scene. Establishing shots have also approached noninitial shot scales, but not their durations. Such results suggest that film form is evolving, perhaps to suit more rapid encoding of narrative events. (shrink)

Narratives are an integral part of human expression. In the graphic form, they range from cave paintings to Egyptian hieroglyphics, from the Bayeux Tapestry to modern day comic books (Kunzle, 1973; McCloud, 1993). Yet not much research has addressed the structure and comprehension of narrative images, for example, how do people create meaning out of sequential images? This piece helps fill the gap by presenting a theory of Narrative Grammar. We describe the basic narrative categories and their relationship to a (...) canonical narrative arc, followed by a discussion of complex structures that extend beyond the canonical schema. This demands that the canonical arc be reconsidered as a generative schema whereby any narrative category can be expanded into a node in a tree structure. Narrative “pacing” is interpreted as a reflection of various patterns of this embedding: conjunction, left-branching trees, center-embedded constituencies, and others. Following this, diagnostic methods are proposed for testing narrative categories and constituency. Finally, we outline the applicability of this theory beyond sequential images, such as to film and verbal discourse, and compare this theory with previous approaches to narrative and discourse. (shrink)

This Document is Protected by copyright and was first published by Frontiers. All rights reserved. it is reproduced with permission.