The aim of this paper is to argue that the phenomenal similarity between perceiving and visualizing can be explained by the similarity between the structure of the content of these two different mental states. And this puts important constraints on how we should think about perceptual content and the content of mental imagery.

Recent research put forward the hypothesis that eye movements are integrated in memory representations and are reactivated when later recalled. However, “looking back to nothing” during recall might be a consequence of spatial memory retrieval. Here, we aimed at distinguishing between the effect of spatial and oculomotor information on perceptual memory. Participants’ task was to judge whether a morph looked rather like the first or second previously presented face. Crucially, faces and morphs were presented in a way that the morph (...) reactivated oculomotor and/or spatial information associated with one of the previously encoded faces. Perceptual face memory was largely influenced by these manipulations. We considered a simple computational model with an excellent match that expresses these biases as a linear combination of recency, saccade, and location. Surprisingly, saccades did not play a role. The results suggest that spatial and temporal rather than oculomotor information biases perceptual face memory. (shrink)

Pupillary responses and associated vergence eye movements were monitored during imagery of objects of differing sizes (“large” or “small”) from varying distances (“near” or “far”). Objects’ imagined size and distance affected oculomotor behavior. Objects visualized as “far” resulted in the larger pupil dilations and smaller visual angle, while small objects imagined “near” were associated with smaller pupils in contrast to relatively larger pupils when imagined as “far” away. Furthermore, near objects resulted in larger visual angle, and particularly, vergence adjustments were (...) observed that were dependent on imagined size and distance of the small objects. The findings are consistent with the idea that imagery emulates perception, also at the embodied level of oculomotor behaviors. (shrink)

People often make use of a spatial “mental time line” to represent events in time. We investigated whether the eyes follow such a mental time line during online language comprehension of sentences that refer to the past, present, and future. Participants' eye movements were measured on a blank screen while they listened to these sentences. Saccade direction revealed that the future is mapped higher up in space than the past. Moreover, fewer saccades were made when two events are simultaneously taking (...) place at the present moment compared to two events that are happening in different points in time. This is the first evidence that oculomotor correlates reflect mental looking along an abstract invisible time line during online language comprehension about time. Our results support the idea that observing eye movements is likely to “detect” invisible spatial scaffoldings which are involved in cognitively processing abstract meaning, even when the abstract meaning lacks an explicit spatial correlate. Theoretical implications of these findings are discussed. (shrink)

Semantic studies on diagrammatic notations (Barwise & Etchemendy, ; Shimojima, ; Stenning & Lemon, ) have revealed that the “non-deductive,” “emergent,” or “perceptual” effects of diagrams (Chandrasekaran, Kurup, Banerjee, Josephson, & Winkler, ; Kulpa, ; Larkin & Simon, ; Lindsay, ) are all rooted in the exploitation of spatial constraints on graphical structures. Thus, theoretically, this process is a key factor in inference with diagrams, explaining the frequently observed reduction of inferential load. The purpose of this study was to examine (...) the empirical basis for this theoretical suggestion, focusing on the reality of the constraint-exploitation strategy in actual practices of diagrammatic reasoning. Eye movements were recorded while participants used simple position diagrams to solve three- or four-term transitive inference problems. Our experiments revealed that the participants could exploit spatial constraints on graphical structures even when (a) they were not in the position of actually manipulating diagrams, (b) the semantic rule for the provided diagrams did not match their preferences, and (c) the constraint-exploitation strategy invited a partly adverse effect. These findings indicate that the hypothesized process is in fact robust, with the potential to broadly account for the inferential advantage of diagrams. (shrink)

Our eyes, bodies, and perspectives are constantly shifting as we observe the world. Despite this, we are very good at distinguishing between self-caused visual changes and changes in the environment: the world appears mostly stable despite our visual field moving around. This, it seems, also occurs when we are dreaming. As we visually investigate the dream environment, we track moving objects with our dream eyes, examine objects, and shift focus. These movements, research suggests, are reflected in the rapid movements or (...) saccades of our sleeping eyes. Do we really see the dream world in the same way that we see the real world? If we do, how could dreaming, usually assumed to be mind-generated hallucinations, replicate such an experience? This problem would be deflated if dreams are not hallucinations at all, but rather imagination, illusion or simply unrealistic. I argue that imagination and illusion views do not satisfactorily explain away the problem of vision and action in sleep. The imagination model is not a complete description of dreaming that is consistent with empirical research, and it is unlikely that the visual dream world is an illusion. Given that the dreaming visual experience is most likely active, hallucinatory, and at times a realistic world simulation, there are important implications for our understanding of visual perception and its relationship to movement. Evidence suggests that our dream eyes investigate the dream world as our waking eyes investigate the waking world. If changes to the unconsciously generated dream environment are perceived as external and unintentional while dream body movements are perceived as self-generated and intentional, current theory of visual perception may have to be expanded to account for how the dreaming mind generates a stable world in which we track and visually explore mind-generated objects. (shrink)

The distinction between perception and cognition has always had a firm footing in both cognitive science and folk psychology. However, there is little agreement as to how the distinction should be drawn. In fact, a number of theorists have recently argued that, given the ubiquity of top-down influences, we should jettison the distinction altogether. I reject this approach, and defend a pluralist account of the distinction. At the heart of my account is the claim that each legitimate way of marking (...) a border between perception and cognition deploys a notion I call ‘stimulus-control.’ Thus, rather than being a grab bag of unrelated kinds, the various categories of the perceptual are unified into a superordinate natural kind. (shrink)

When we see an object, we also represent those parts of it that are not visible. The question is how we represent them: this is the problem of amodal perception. I will consider three possible accounts: (a) we see them, (b) we have non-perceptual beliefs about them and (c) we have immediate perceptual access to them, and point out that all of these views face both empirical and conceptual objections. I suggest and defend a fourth account, according to which we (...) represent the occluded parts of perceived objects by means of mental imagery. This conclusion could be thought of as a (weak) version of the Strawsonian dictum, according to which “imagination is a necessary ingredient of perception itself”. (shrink)

Three studies were conducted to examine the relation of spatial visualization to solving kinematics problems that involved either predicting the two‐dimensional motion of an object, translating from one frame of reference to another, or interpreting kinematics graphs. In Study 1, 60 physics‐naíve students were administered kinematics problems and spatial visualization ability tests. In Study 2, 17 (8 high‐ and 9 low‐spatial ability) additional students completed think‐aloud protocols while they solved the kinematics problems. In Study 3, the eye movements of fifteen (...) (9 high‐ and 6 low‐spatial ability) students were recorded while the students solved kinematics problems. In contrast to high‐spatial students, most low‐spatial students did not combine two motion vectors, were unable to switch frames of reference, and tended to interpret graphs literally. The results of the study suggest an important relationship between spatial visualization ability and solving kinematics problems with multiple spatial parameters. (shrink)

The neural correlates of rapid eye movements (REMs) in sleep are extraordinarily robust; including REM-locked activation in the retrosplenial cortex, the supplementary eye field and areas overlapping cholinergic basal nucleus. The phenomenology of REMs speaks to the notion that perceptual experience in both sleep and wakefulness is a constructive process – in which we generate predictions of sensory inputs and then test those predictions through actively sampling the sensorium with eye movements. On this view, REMs during sleep may index an (...) internalized active sampling or ‘scanning’ of self-generated visual constructs that are not constrained by visual input. If this view is correct, it renders REMs an ideal probe to study consciousness as “an exclusively internal affair” (Metzinger 2009). In other words, REMs offer a probe of active inference – in the sense of predictive coding – when the brain is isolated from the sensorium in virtue of the natural blockade of sensory input during REM sleep. REMs are temporally precise events that enable powerful inferences based on timeseries analyses. As a natural, task-free probe, [REMs] could be used in non-compliant subjects, including infants and animals. In short, REM constitutes a promising probe to study the ontogenetic and phylogenetic development of consciousness and perhaps its abnormal development in schizophrenia and autism, which have been considered in terms of aberrant predictive coding. (shrink)

Traditionally, the “Imagery Debate” has opposed two main camps: depictivism and descriptivism. This debate has essentially focused on the nature of the internal representations thought to be involved in imagery, without addressing at all the question of action. More recently, a third, “embodied” view is moving the debate into a new phase. The embodied approach focuses on the interdependence of perception, cognition and action, and in its more radical line this approach promotes the idea that perception is not a process (...) involving internal world-models. The anti-representationalist version of the embodied paradigm covers, among others that we shall not discuss here, two quite different positions, namely the enactive approach and sensorimotor theory. Up to now these two anti-representationalist accounts have generally been confounded. In this paper we will argue that despite some important commonalities, enactive and sensorimotor accounts come with distinctive theoretical traits with regard to their approach to imagery. These become manifest when critically examining the role they assign to sensorimotor engagements with the world. We shall argue that enactive and sensorimotor approaches are different in their understanding of the role of embodied action, and these different notions of embodiment lead to different explanatory accounts of perception and imagery. We propose that, due to existing ambiguities in enactivism, the sensorimotor theory is a better framework for a skill-based approach to imagery. (shrink)

Studies on mind-wandering frequently use reading as an experimental task. In these studies, reading is conceived as a cognitive process that potentially offers a contrast to mind-wandering, because it seems to be task-related, goal-directed and stimulus-dependent. More recent work attempts to avoid the dichotomy of successful cognitive processes and processes of mind-wandering found in earlier studies. We approach the issue from the perspective that texts provoke modes of cognitive involvement different from information processing and recall account that underlies many early (...) mind-wandering studies. After all, reading itself is an umbrella term for activities that are characterised by a variety of phenomenological and functional signatures. We conceptualise reading and mind-wandering in relation to each other through the framework of enculturated predictive processing, which is informed by research in literary studies. Earlier accounts think of reading predominantly in terms of the construction of situation models to organise textual information. By contrast, enculturated predictive processing foregrounds shifting stances readers can take in relation to the text. Characters featuring in literary texts might be mind-wandering themselves, or passages in literary style might make the construction of a clearly defined situational model impossible. Furthermore, we take into account that texts often elicit mind-wandering through the construction of task-relevant and attention-driven virtual scenarios in readers’ minds. This more complex account of reading can enrich recent attempts to understand mind-wandering as a complex, multi-dimensional phenomenon. The study of mind-wandering can benefit, we argue, from a closer attention to the process of reading and to the texts it deploys as stimuli. The emerging perspective from enculturated predictive processing and literary studies makes distinctions in reading that in turn enable research on mind-wandering to ask more precise questions about (1) different kinds of mind-wandering, (2) different modes of reading, and (3) how and where they interconnect across time. (shrink)

One characteristic of the transition from the Middle Paleolithic to the Upper Paleolithic in Europe was the emergence of representational charcoal drawings and engravings by Aurignacian and Gravettian artists. European Neanderthals never engaged in representational drawing during the Middle and Early Upper Paleolithic, a property that might reflect less developed visuomotor coordination. This article postulates a causal relationship between an evolved ability of anatomically modern humans to throw spears accurately while hunting and their ability to draw representational images from working (...) memory. Unlike Neanderthals, archaic and anatomically modern humans in sub-Saharan Africa adopted longer-range hunting practices using hand-cast spears as a compensation for the emergence of increasingly wary game. For Neanderthals, paleoclimatic fluctuations likely precluded consistent hunting of cold-adapted game, a property making game more approachable for close-range hunting with thrusting spears. As evidence of less historical wariness of humans, many of the species hunted by Neanderthals were eventually domesticated. Due to strong sources of natural selection on archaic and anatomically modern humans for effective hunting, the parietal cortex that integrates visual imagery and motor coordination expanded progressively, yielding the globular shape of the human cranium that is not evident in Neanderthals. To characterize how the cognitive properties employed for throwing spears and drawing line work are similar, the Upper Paleolithic drawings of animals in Chauvet cave, France, are discussed in the speculative context of how these artists engaged simultaneously in overt attention to guide their hand movements and covert attention to their mental images during the drawing process. (shrink)

In this paper, I will examine the question of the space of visual imagery. I will ask whether in visually imagining an object or a scene, we also thereby imagine that object or scene as being in a space unrelated to the space we’re simultaneously perceiving or whether it is the case that the space of visual imagination is experienced as connected to the space of perceptual experience. I will argue that the there is no distinction between the spatial content (...) of visualization and the spatial content of visual perception. I will base my conclusion on two uncontroversial, empirically confirmed aspects of imagery: the perspectival character of imagery, and the possibility of superimposing an imagined object upon the perceived scene. (shrink)

We identify a number of principles with respect to prediction that, we argue, underpin adult language comprehension: (a) comprehension consists in realizing a mapping between the unfolding sentence and the event representation corresponding to the real‐world event being described; (b) the realization of this mapping manifests as the ability to predict both how the language will unfold, and how the real‐world event would unfold if it were being experienced directly; (c) concurrent linguistic and nonlinguistic inputs, and the prior internal states (...) of the system, each drive the predictive process; (d) the representation of prior internal states across a representational substrate common to the linguistic and nonlinguistic domains enables the predictive process to operate over variable time frames and variable levels of representational abstraction. We review empirical data exemplifying the operation of these principles and discuss the relationship between prediction, event structure, thematic role assignment, and incrementality. (shrink)

Is an appropriate causal connection to the past experience it represents a necessary condition for a mental state to qualify as an episodic memory? For some years this issue has been the subject of an intense debate between the causalist theory of episodic memory (CTM) and the simulationist theory of episodic memory (STM). This paper aims at exploring the prospects for an embodied approach to episodic memory and assessing the potential case for causalism that could be founded on it. In (...) a critical section, it argues that the empirical data to which STM appeals are both incomplete and inconclusive, and on closer examination even provide support for a proceduralist version of CTM. In a constructive section, it elaborates on the notion of a necessary causal connection in terms of particular procedural patterns acquired at encoding and operative at retrieval, grounding this move on recent empirical data about eye movements in mnemonic mental imagery. (shrink)

We are aware of those parts of a cat that are occluded behind a fence. The question is how we represent these occluded parts of perceived objects: this is the problem of amodal perception. I will consider four theories and compare their explanatory power: (i) we see them, (ii) we have nonperceptual beliefs about them, (iii) we have immediate perceptual access to them and (iv) we visualize them. I point out that the first three of these views face both empirical (...) and conceptual objections. I argue for the fourth account, according to which we visualize the occluded parts of perceived objects. Finally, I consider some important consequences of this view with regards to the content and the evolution of visualization. (shrink)

This thesis investigates the relationship between eye movements, mental imagery and memory retrieval in four studies based on eye-tracking experiments. The first study is an investigation of eye movements during mental imagery elicited both visually and verbally. The use of complex stimuli and the development of a novel method where eye movements are recorded concurrently with verbal data enabled the above-mentioned relationship to be studied to an extent going beyond what previous research had been able to do. Eye movements were (...) found to closely reflect content and spatial layout while participants were listening to a spoken scene description, while they were describing the same scene from memory, and while they were describing a picture they had previously seen. This effect was equally strong during recall from memory irrespective of whether the scene visualised had originally been inspected visually by the participants or whether it was constructed whole-cloth from long-term memory. It was also found that eye movements "to nothing" appeared both when the participants were visualising scenes while looking at a blank screen and when they were doing so in complete darkness. The second study explored an effect frequently observed in the first study, involving a "scaling-down" during recall of participants' gaze patterns to an area smaller than that occupied by the stimulus encoded. It was found that this scaling effect correlated with spatial-imagery ability: the gaze patterns of participants with weaker spatial-imagery ability were closer in size to the encoded scene than the gaze patterns of those stronger in spatial-imagery ability. In the third study, the role of eye movements during mental imagery was investigated in four experiments where eye movements were prohibited during either the encoding phase or the recall phase. Experiments 1 and 2 showed that maintaining central fixation during visual or auditory encoding, respectively, had no effect on how eye movements were executed during recall. Thus, oculomotor events during recall are not reproductions of those produced during encoding. In Experiments 3 and 4, central fixation was instead maintained during recall. This turned out to alter and impair scene recollection, irrespective of the modality of encoding. Finally, in the fourth study, the functional role of eye movements in relation to memory retrieval was further investigated by means of direct eye-movement manipulation in the retrieval phase of an episodic-memory task. Four conditions were used: free viewing on a blank screen, maintaining central fixation, viewing within a square congruent with the location of the objects to be recalled, and viewing within a square incongruent with the location of the objects to be recalled. The results obtained show that gaze position plays an active and facilitatory role during memory retrieval. The findings from these studies are discussed in the light of current theories regarding eye movements during mental imagery and memory retrieval. (shrink)

A theory of the structure and cognitive function of the human imagination that attempts to do justice to traditional intuitions about its psychological centrality is developed, largely through a detailed critique of the theory propounded by Colin McGinn. Like McGinn, I eschew the highly deflationary views of imagination, common amongst analytical philosophers, that treat it either as a conceptually incoherent notion, or as psychologically trivial. However, McGinn fails to develop his alternative account satisfactorily because (following Reid, Wittgenstein and Sartre) he (...) draws an excessively sharp, qualitative distinction between imagination and perception, and because of his flawed, empirically ungrounded conception of hallucination. His arguments in defense of these views are rebutted in detail, and the traditional, passive, Cartesian view of visual perception, upon which several of them implicitly rely, is criticized in the light of findings from recent cognitive science and neuroscience. It is also argued that the apparent intuitiveness of the passive view of visual perception is a result of mere historical contingency. An understanding of perception (informed by modern visual science) as an inherently active process enables us to unify our accounts of perception, mental imagery, dreaming, hallucination, creativity, and other aspects of imagination within a single coherent theoretical framework. (shrink)