What is the correct procedure for determining the contents of perception? Philosophers tackling this question increasingly rely on empirically-oriented procedures in order to reach an answer. I argue that this constitutes an improvement over the armchair methodology constitutive of phenomenal contrast cases, but that there is a crucial respect in which current empirical procedures remain limited: they are unimodal in nature, wrongly treating the senses as isolatable faculties. I thus have two aims: first, to motivate a reorientation of the (...) admissible contents debate into a multimodal framework, charting its various significances. The second is to explore whether any experimental studies of multimodal perception support a so-called Liberal (or ‘high-level’ or ‘rich’) account of perception’s admissible contents. I conclude that the McGurk effect and the ventriloquist effect are both explicable without the postulation of high-level content, but that at least one multimodal experimental paradigm may necessitate such content: the rubber hand illusion. One upshot of this argument is that Conservatives who claim that the Liberal view intolerably broadens the scope of perceptual illusions, particularly from the perspective of perceptual psychology, should pursue other arguments against that view. (shrink)

Higher animals need to identify and track material objects because they depend on interactions with them for nutrition, reproduction, and social interaction. This paper investigates the perception of material objects. It argues, first, that material objects are tagged, in all five external senses, as bearers of the features detected by them. This happens through a perceptual process, here entitled Generalized Completion, which creates the appearance of objects that have properties that transcend the activation of sensory receptors. The paper shows, (...) secondly, that material objects are privileged subjects for perceived motion and interaction. That is, they are perceived as subjects for these properties while their parts seem to be subjects only derivatively. Material objects are the only perceptual subjects that are both multisensory and privileged. (shrink)

Stoffregen & Bardy reject the likelihood that infants are sensitive to the global array, implying that intersensory integration is not possible in early development. We argue that infants are sensitive to unimodal arrays and are able to integrate them through the active participation of their nervous system and that the observed developmental changes are due to experience and brain development.

Philosophers and cognitive scientists of perception by custom have investigated individual sense modalities in relative isolation from each other. However, perceiving is, in a number of respects, multimodal. The traditional sense modalities should not be treated as explanatorily independent. Attention to the multimodal aspects of perception challenges common assumptions about the content and phenomenology of perception, and about the individuation and psychological nature of sense modalities. Multimodal perception thus presents a valuable opportunity for (...) a case study in mature interdisciplinary cognitive science. This chapter aims to raise these issues against the background of unimodal approaches in the study of perception. It presents some of the central empirical findings concerning multimodality, and it explains the philosophical implications of these findings. Foremost, it aims to encourage and open avenues for future research. (shrink)

Everyday media consumption leaves no doubt that the perception of moving images from various media is characterized by experience and understanding. Corresponding research in this field has shown that the stimulus patterns flooding in on us are not only processed mentally, but also bodily. Building on this, the following study argues that incoming stimuli are processed not only visually, but multimodally, with all senses, and moreover affectively. The classical binding of a sensory organ to a medium, on whose delimitation (...) the common understanding of multimodality is based, is thus abolished. Can arguments for this assumption be found? Building on the research of cultural anthropologist Ernst Cassirer and his successors, the two American philosophers Susanne K. Langer and John M. Krois, and supported by research from the life sciences in which, in parallel, the arts also play an important role in understanding image formation, as in the work of Heinz Werner and, following on from this, in the more recent research of the child psychologist Daniel N. Stern in collaboration with mirror neuron research by the group of neuroscientists around Giacomo Rizzolatti in Parma, justifications for this will be provided. What the multimodal processing of film images is concretely based on will finally be made clear by means of an exemplary analysis of a video film by the Swiss artist Pipilotti Rist. It aims to prove, that the prerequisites for the thesis can only lie in the fact, that both the modes of perception (visual, tactile, auditory, olfactory, gustatory) as well as the medially conditioned modes of image formation are based on non-discursive, formal-abstract structures that are also processed affectively. At the same time, the analysis thus shows how the selection and composition of the vital experienced forms can provide information about the socio-culturally shaped self-image of the producer and the viewer. (shrink)

When I am looking at my coffee machine that makes funny noises, this is an instance of multisensory perception – I perceive this event by means of both vision and audition. But very often we only receive sensory stimulation from a multisensory event by means of one sense modality, for example, when I hear the noisy coffee machine in the next room, that is, without seeing it. The aim of this paper is to bring together empirical findings about (...) class='Hi'>multimodal perception and empirical findings about (visual, auditory, tactile) mental imagery and argue that on occasions like this, we have multimodal mental imagery: perceptual processing in one sense modality (here: vision) that is triggered by sensory stimulation in another sense modality (here: audition). Multimodal mental imagery is not a rare and obscure phenomenon. The vast majority of what we perceive are multisensory events: events that can be perceived in more than one sense modality – like the noisy coffee machine. And most of the time we are only acquainted with these multisensory events via a subset of the sense modalities involved – all the other aspects of these multisensory events are represented by means of multisensory mental imagery. This means that multisensory mental imagery is a crucial element of almost all instances of everyday perception. (shrink)

The aim of this paper is to argue that our experience of artworks is normally multimodal. It is the result of perceptual processing in more than one sense modality. In other words, multimodal experience of art is not the exception; it is the rule. I use the example of music in order to demonstrate the various ways in which the visual sense modality influences the auditory processing of music and conclude that this should make us look more closely (...) at our practices of engaging with artworks. (shrink)

Sensory Individuals: Unimodal and Multimodal Perspectives provides an interdisciplinary, well-balanced, and comprehensive look at different aspects of unisensory and multisensory objects, using both nuanced philosophical analysis and informed empirical work. The research presented in this book represents the field's progression from treating neural sensory processes as primarily modality-specific towards its current state of the art, according to which perception, and its supporting neural processes, are multi-modal, modality-independent, meta-modal, and task-dependent. Even within such approaches sensory stimuli, properties, brain activations, (...) and corresponding perceptual phenomenology can still be characterized in a modality-specific way. The book examines the basic building blocks of human perception, and whether they are best understood as sense modality dependent units of different forms or multimodal perceptual objects. The book combines a variety of innovative and integrative angles to explore the topic and acts as a catalyst for an increasingly diverse field of research, which is in an exciting phase of growth and advancement. New questions are arising as quickly as they are being answered, and the collection Sensory Individuals provides an original and up-to-date addition to the field. (shrink)

Many philosophers use findings about sensory substitution devices in the grand debate about how we should individuate the senses. The big question is this: Is “vision” assisted by (tactile) sensory substitution really vision? Or is it tactile perception? Or some sui generis novel form of perception? My claim is that sensory substitution assisted “vision” is neither vision nor tactile perception, because it is not perception at all. It is mental imagery: visual mental imagery triggered by tactile (...) sensory stimulation. But it is a special form of mental imagery that is triggered by corresponding sensory stimulation in a different sense modality, which I call “multimodal mental imagery.”. (shrink)

Sensory Individuals: Unimodal and Multimodal Perspectives provides an interdisciplinary, well-balanced, and comprehensive look at different aspects of unisensory and multisensory objects, using both nuanced philosophical analysis and informed empirical work. The research presented in this book represents the field's progression from treating neural sensory processes as primarily modality-specific towards its current state of the art, according to which perception, and its supporting neural processes, are multi-modal, modality-independent, meta-modal, and task-dependent. Even within such approaches sensory stimuli, properties, brain activations, (...) and corresponding perceptual phenomenology can still be characterized in a modality-specific way. The book examines the basic building blocks of human perception, and whether they are best understood as sense modality dependent units of different forms or multimodal perceptual objects. The book combines a variety of innovative and integrative angles to explore the topic and acts as a catalyst for an increasingly diverse field of research, which is in an exciting phase of growth and advancement. New questions are arising as quickly as they are being answered, and the collection Sensory Individuals provides an original and up-to-date addition to the field. (shrink)

Analysing film. Distinguishing the filmic contribution to meaning -- Examples of filmic "textual organisation" -- Redrawing boundaries -- Organisation of the book -- Semiotics and documents. Semiotics and its relations to film -- The nature of discourse semantics -- The film as cinematographic document -- A combined view: filmic documents for filmic discourse -- Constructing the semiotic mode of film. Semiotic multimodality -- The internal organisation of semiotic strata -- Composing and combining semiotic modes -- Materiality and "epistemological commitment" -- (...) Christian Metz and the grande syntagmatique of the image track. The original model -- Two examples of analysis with the grande syntagmatique -- Revisions and rebuttals -- Foundations for analysis: filmic units. The basic units of film: preliminaries -- Audiovisual iconic representations -- Perception, perceptual realism and reliable measurement -- Multiplicity: from perception to discourse -- Filmic units revisited: discourse-motivated definitions -- The paradigmatic organisation of film. Beyond Metz: towards a grande paradigmatique -- Capturing discourse dependency structures in film -- The paradigmatic dimensions of projection, taxis and plane -- Two examples of paradigmatic analysis -- Summary and conclusions. (shrink)

Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker’s face. In the critical condition, the visual (e.g. /gi/) (...) and auditory (e.g. /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g. /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants’ ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning. (shrink)

There has been a lot of discussion about how the cognitive penetrability of perception may or may not have important implications for understanding perceptual justification. The aim of this paper is to argue that a different set of findings in perceptual psychology poses an even more serious challenge to the very idea of perceptual justification. These findings are about the importance of perceptual processing that is not driven by corresponding sensory stimulation in the relevant sense modality (such as amodal (...) completion and multimodal completion). I argue that these findings show that everyday perception is in fact a mixture of sensory stimulation-driven perceptual processing and perceptual processing that is not driven by corresponding sensory stimulation in the relevant sense modality and we have strong reasons to doubt the epistemic pedigree of the latter process. The implication of this is not that we should become skeptics or deny the possibility of perceptual justification. It is, rather, that the only way in which we can understand when and whether a perceptual state justifies beliefs is by paying close attention to empirical facts about the reliability of perceptual processing that is not driven by corresponding sensory stimulation in the relevant sense modality. In this sense (a very narrow sense) epistemology needs to be naturalized. (shrink)

This paper explores a semiotic notion of body as starting point for bridging biosemiotic with social semiotic theory. The cornerstone of the argument is that the social semiotic criticism of the classic view of meaning as double articulation can support the criticism of language-centrism that lies at the foundation of biosemiotics. Besides the pragmatic epistemological advantages implicit in a theoretical synthesis, I argue that this brings a semiotic contribution to philosophy of mind broadly. Also, it contributes to overcoming the polemic (...) in linguistics between, loosely put, cognitive universalism and cultural relativism. This possibility is revealed by the recent convergence of various semiotic theories towards a criticism of the classic notion of meaning as double articulation. In biosemiotics, the interest to explicate meaning as multiply articulated stems from the construal of Umwelt as relying on the variety of sense perception channels and semiotic systems that a species has at its disposal. Recently, social semiotics developed an unexplored interest for embodiment by starting from the other end, namely the consideration of the modal heterogeneity of meaning. To bridge these notions, I employ the cognitive semantic notion of embodiment and Mittelberg’s cognitive semiotic notion of exbodiment. In light of these, I explore the possible intricacies between the biosemiotic notion of primary modeling system and concepts referring to preconceptual structures for knowledge organization stemming from cognitive linguistics. Further, Mittelberg’s concept of exbodiment allows for a construal of meaning articulation as mediation between the exbodying and embodying directions of mind. (shrink)

This article starts with the environmental changes in human cognition, analyzes the virtual as the main feature of visual perception under digital technology, and explores the transition from passive to active human cognitive activities. With the diversified understanding of visual information, human contradiction of memory also began to become prominent. Aiming at the problem that the existing multimodal TV media recognition methods have low recognition rate of unknown application layer protocols, an adaptive clustering method for identifying unknown application (...) layer protocols is proposed. This method clusters application layer protocols based on similarity of the load characteristics of network stream application layer protocol data. The method divides the similarity calculation in the clustering algorithm to improve the clustering efficiency of the algorithm. Experimental results show that the proposed method can efficiently and accurately recognize unknown visual communication. This article proposes that, in the interactive multimodal visual information transmission, human visual perception experience has changed, the diversity of visual information content expression makes the aesthetic subject more personalized and stylized. (shrink)

This is an excerpt of a report that highlights and explores five questions which arose from The Unity of Consciousness and Sensory Integration conference at Brown University in November of 2011. This portion of the report explores the question: Are some of the basic units of consciousness multimodal?

Mental imagery is early perceptual processing that is not triggered by corresponding sensory stimulation in the relevant sense modality. Multimodal mental imagery is early perceptual processing that is triggered by sensory stimulation in a different sense modality. For example, when early visual or tactile processing is triggered by auditory sensory stimulation, this amounts to multimodal mental imagery. Pulling together philosophy, psychology and neuroscience, I will argue in this paper that multimodal mental imagery plays a crucial role in (...) our engagement with musical works. Engagement with musical works is normally a multimodal phenomenon, where we get input from a number of sense modalities. But even if we screen out any input that is not auditory, multimodal mental imagery will still play an important role that musicians and composers often actively rely on. (shrink)

An introduction to contemporary debates in philosophy of perception.

This is the editors' Introduction to a special issue of the journal, Multisensory Research. European philosophers of the modern period found multisensory perception to be impossible because they thought that perceptual ideas are defined by how they are experienced. Under this conception, the individual modalities are determinables of ideas—just as colour is a determinable that embraces red and blue, so also the visual is a determinable that embraces colour and (visually experienced) shape. Since no idea is experienced as, for (...) example, both visual and auditory, there can be no such thing as audiovisual perception. This conception of modality is not directly contested, but a variety of perceptual phenomena are listed that could raise interesting questions if treated as multimodal in origin. (shrink)

This collection of new essays on sensory individuals in unimodal and multimodal perception features contributions by outstanding researchers in the fields of philosophy of perception, experimental psychology, and cognitive neuroscience. The topics investigated include conceptual, developmental, and methodological aspects of object perception, and especially how various sense modalities construct their objects from sensory features and feature bearers. The interdisciplinary approach offered has enabled new directions in research on this subject. As ordered in this volume, the topics (...) of the contributions progress from treating neural sensory processes as primarily unisensory towards approaches that take perceptual processing to be modality-independent, meta-modal, and multimodal. Even within these latter approaches, sensory stimuli, sensory properties, brain activations, and corresponding perceptual phenomenology have often been characterized in a modality-specific way. Thus, it is timely to explore the relation between those processes that are unisensory and those that are multisensory. One of the primary themes of this collection is examining whether the basic building blocks of human perception are best understood as modality-dependent units of different forms or in terms of multimodal perceptual objects. Another theme is the relation between low-level object processing (segmentation and perceptual grouping processes) and high-level object knowledge embracing object concept and object-related expectations. A final theme is the role that perceptual objects play as loci of unification in unimodal and multimodal perception, namely that they enable binding and integration of sensory properties to individual entities or events. (shrink)

Although multimodal input has the potential to lead to more sound learning outcomes, it carries the risk of causing cognitive overload, making it difficult to determine the exact effects of multimodal input on the second language phrase learning. This study tests the efficacy of multimodal input on L2 phrase learning. It adopts a mixed-method approach by utilizing both quantitative and qualitative data. The experimental design is a 2 × 3 mixed model, with a group [the experimental group (...) and the control group ] as the between-subject factor and time as the within-subject factor. A total of 66 participants were divided into two groups. All materials incorporated three aspects of phrase knowledge, but the materials of the CG were unimodal in that they were offered only on paper, and of the EG were multimodal in that they included pictures, audio recordings, and video clips. After the treatment, a questionnaire and a semi-structured interview were given to the EG learners to explore their perceptions of using multimodal materials to learn L2 phrases. The results indicate that both groups had significant gains in learning phrases, but students with the multimodal input achieved significantly better results than those with the unimodal input. Moreover, the EG students had a generally positive attitude toward the use of multimodal resources. This study validates the efficacy of multimodal input on the acquisition of English phrases and shows that cognitive overload was avoided by sequencing the information. (shrink)

Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O’Callaghan (forthcoming) calls this awareness “intermodal feature binding awareness.” Psychologists have long assumed that multimodal perceptions such as this one are the result of a subpersonal feature binding (...) mechanism (see Vatakis and Spence, 2007, Kubovy and Schutz, 2010, Pourtois et al., 2000, and Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal’s jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called “unitization,” whereby we come to “chunk” the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux’s question to the question of how active or passive we consider perception to be. (shrink)

In this contribution we will approach language and images in different modalities: speech and face, anticipated and imagined movements, illusions on the sound by the image. It will be the opportunity for us to revisit the Gestalt concepts which were considered obsolete since structuralism in Humanities. As instantiated by Gilbert Durand in The Anthropological Structures of the Imaginary (1999, French 1st ed. 1960), we shall recall that Gestalt is not—even implicitly—an exclusively static approach to cognition. On the contrary we will (...) emphasize that forms can emerge from movements and stabilize in memory, thence morphology. And for speech, we will consider phonology as the stabilized outcome from audible and visible gestures of the mouth—caught in the time-course of a flow of some four in six syllables by second—via the perception of motor coordinations: which is necessary to master the most common expression of language between humans. We will adopt here also for the perceptual flows of spoken language, Multimodal Scene Analysis, the ongoing legacy of Gestalt Theory, including one of our main research topics: the gestural expression of the face and the hand for Deaf people, who practise as an augment French LPC (Langue française Parlée Complétée), adapted in 1967 from Cued Speech by its originator Dr Richard Cornett. (shrink)

Is it true that there is an analogy between modes of creation and such of perception? Respective to the cultural anthropological research of Ernst Cassirer, Susanne K. Langer and John M. Krois and by the analysis of a tape of the Swiss video-artist Pipilotti Rist this initial thesis of Formal Aesthetics shall be supported. - I - -/- Lässt sich die Behauptung stützen, dass zwischen Gestaltungsweisen und Wahrnehmungsweisen eine Analogie besteht? Aufbauend auf den kulturanthropologischen Forschungen von Ernst Cassirer, Susanne (...) K. Langer und John M. Krois sowie der Analyse eines Tapes der Schweizer Videokünstlerin Pipilotti Rist gilt es diese ursprünglich von der formalen Ästhetik aufgeworfene These zu untermauern. (shrink)

Experimental investigations of cross-cultural music perception and cognition reported during the past decade are described. As globalization and Western music homogenize the world musical environment, it is imperative that diverse music and musical contexts are documented. Processes of music perception include grouping and segmentation, statistical learning and sensitivity to tonal and temporal hierarchies, and the development of tonal and temporal expectations. The interplay of auditory, visual, and motor modalities is discussed in light of synchronization and the way music (...) moves via emotional response. Further research is needed to test deep-rooted psychological assumptions about music cognition with diverse materials and groups in dynamic contexts. Although empirical musicology provides keystones to unlock musical structures and organization, the psychological reality of those theorized structures for listeners and performers, and the broader implications for theories of music perception and cognition, awaits investigation. (shrink)

One of the recent areas of discussion in aesthetics and the visual arts is the tension between the so-called “ocularcentric” tradition, on the one hand, and the tendency to move in a multisensory, multimodal direction, on the other. My aim in this paper is to bring out this tension by tracing it in a number of moments; firstly, in the late 19th-early 20th century discussion, concerning the “total art work” and the contribution of synaesthesia; secondly, the reaction to what (...) Clement Greenberg called the modernist “reduction to the visual”, expressed in a variety of multimodal art forms in the latter half of the 20th century. The weight of my paper is given to two contemporary discussions with significant repercussions in aesthetics: a. the opposition between “disembodiment” and “embodied immersion” in the area of new media; and b. the debate between modularity and plasticity of perception in neuroscience. The conclusion to which I am led is that the possibilities of sensory integration offered by new media and of sensory plasticity - offered by synaesthesia - suggest new creative ways in which our senses may interact and enhance our experience of the world, as well as elucidate artistic creativity and aesthetic appreciation. (shrink)

Atypical sensorimotor developmental trajectories greatly contribute to the profound heterogeneity that characterizes Autism Spectrum Disorders. Individuals with ASD manifest deviations in sensorimotor processing with early markers in the use of sensory information coming from both the external world and the body, as well as motor difficulties. The cascading effect of these impairments on the later development of higher-order abilities underlines the need for interventions that focus on the remediation of sensorimotor integration skills. One of the promising technologies for such stimulation (...) is Immersive Virtual Reality. In particular, head-mounted displays have unique features that fully immerse the user in virtual realities which disintegrate and otherwise manipulate multimodal information. The contribution of each individual sensory input and of multisensory integration to perception and motion can be evaluated and addressed according to a user’s clinical needs. HMDs can therefore be used to create virtual environments aimed at improving people’s sensorimotor functioning, with strong potential for individualization for users. Here we provide a narrative review of the sensorimotor atypicalities evidenced by children and adults with ASD, alongside some specific relevant features of IVR technology. We discuss how individuals with ASD may interact differently with IVR versus real environments on the basis of their specific atypical sensorimotor profiles and describe the unique potential of HMD-delivered immersive virtual environments to this end. (shrink)

The basis of perception is the processing and categorization of perceptual stimuli from the environment. Much progress has been made in the science of perceptual categorization. Yet there is still no consensus on how the brain generates sensory individuals, from sensory input and perceptual categories in memory. This chapter argues that perceptual categorization is highly variable across perceivers due to their use of different perceptual strategies for solving perceptual problems they encounter, and that the perceptual system structurally adjusts to (...) the strategies that are most successful. Despite this variability, the different sensory individuals that result when different perceivers rely on different strategies nonetheless correspond to actual external world objects/features. This view is called 'perceptual pluralism'. (shrink)

The study of a middle school social studies and literacy project this paper addresses occurred in the national capital region of the United States, where perceptions of “patriotism” and immigration policies were the subjects of frequent media reports. With this examination the authors considered one overarching research question: how do middle school students describe and illustrate citizenship when given access to multimodal texts and media (e.g., digital photography and slam poetry)? The authors called on young adolescents to create slam (...) poems with incorporated images to address the question “What does it mean to be a ‘citizen’?”. The authors examined products of this project, which included surveys and slam poems, to address this research question, identifying findings that might inform social studies educators’ practices and curricula. Employing alternative and multimodal texts motivated students to engage with this project and to more candidly share expansive perspectives about what they believe makes a “citizen”. (shrink)

Multimodal integration is the formation of a coherent percept from different sensory inputs such as vision, audition, and somatosensation. Most research on multimodal integration in speech perception has focused on audio-visual integration. In recent years, audio-tactile integration has also been investigated, and it has been established that puffs of air applied to the skin and timed with listening tasks shift the perception of voicing by naive listeners. The current study has replicated and extended these findings by (...) testing the effect of air puffs on gradations of voice onset time along a continuum rather than the voiced and voiceless endpoints of the original work. Three continua were tested: bilabial (“pa/ba”), velar (“ka/ga”), and a vowel continuum (“head/hid”) used as a control. The presence of air puffs was found to significantly increase the likelihood of choosing voiceless responses for the two VOT continua but had no effect on choices for the vowel continuum. Analysis of response times revealed that the presence of air puffs lengthened responses for intermediate (ambiguous) stimuli and shortened them for endpoint (non-ambiguous) stimuli. The slowest response times were observed for the intermediate steps for all three continua, but for the bilabial continuum this effect interacted with the presence of air puffs: responses were slower in the presence of air puffs, and faster in their absence. This suggests that during integration auditory and aero-tactile inputs are weighted differently by the perceptual system, with the latter exerting greater influence in those cases where the auditory cues for voicing are ambiguous. (shrink)

Textual and visual analyses of nation-branding campaigns are rare but highly needed (Bolin and Ståhlberg, 2010; Hao, Paul, Trott, Guo, and Wu, 2019) as online media have become a popular tool for states to shape people’s perception (Volcic and Andrejevic, 2011). In Anholt’s much applied nation brand hexagon (2007), immigration and investment, society, governance, and culture and heritage are, along with tourism and export, the core aspects that build a country’s reputation. As the 2015 refugee peak situation resulted in (...) a more restrictive approach of the Swedish government towards asylum applicants, the country’s brand was put under pressure. How could Sweden’s values related to openness be highlighted while policymakers chose a deterrent road? In this study, we bridge streams of research on nation branding, framing, and migration studies by presenting a multimodal analysis of the “Portraits of migration” campaign as a strategic response to the refugee situation. (shrink)

While influential works since the 1970s have widely assumed that imitation is an innate skill in both human and non-human primate neonates, recent empirical studies and meta-analyses have challenged this view, indicating other forms of reward-based learning as relevant factors in the development of social behavior. The visual input translation into matching motor output that underlies imitation abilities instead seems to develop along with social interactions and sensorimotor experience during infancy and childhood. Recently, a new visual stream has been identified (...) in both human and non-human primate brains, updating the dual visual stream model. This third pathway is thought to be specialized for dynamics aspects of social perceptions such as eye-gaze, facial expression and crucially for audio-visual integration of speech. Here, we review empirical studies addressing an understudied but crucial aspect of speech and communication, namely the processing of visual orofacial cues and its integration with vocal auditory cues. Along this review, we offer new insights from our understanding of speech as the product of evolution and development of a rhythmic and multimodal organization of sensorimotor brain networks, supporting volitional motor control of the upper vocal tract and audio-visual voices-faces integration. (shrink)

The human experience is shaped by information from different perceptual channels, but it is still debated whether and how differential experience influences language use. To address this, we compared congenitally blind, blindfolded, and sighted people's descriptions of the same motion events experienced auditorily by all participants (i.e., via sound alone) and conveyed in speech and gesture. Comparison of blind and sighted participants to blindfolded participants helped us disentangle the effects of a lifetime experience of being blind versus the task-specific effects (...) of experiencing a motion event by sound alone. Compared to sighted people, blind people's speech focused more on path and less on manner of motion, and encoded paths in a more segmented fashion using more landmarks and path verbs. Gestures followed the speech, such that blind people pointed to landmarks more and depicted manner less than sighted people. This suggests that visual experience affects how people express spatial events in the multimodal language and that blindness may enhance sensitivity to paths of motion due to changes in event construal. These findings have implications for the claims that language processes are deeply rooted in our sensory experiences. (shrink)

Within contemporary philosophy of perception, it is commonly claimed that flavour experiences are paradigmatic examples of multimodal perceptual experiences. In fact, virtually any sensory system, including vision and audition, is believed to influence how we experience flavours. However, there is a strong intuition, often expressed in these works, that not all of these sensory systems make an equal contribution to the phenomenology of flavour experiences. More specifically, it seems that the activities of some sensory systems are constitutive for (...) flavour perception while others merely influence how we experience flavours. This paper aims to answer the question regarding the constitutive factors of flavour perception in a twofold way. First, a theoretical framework is developed, relying on debates regarding constitutivity in analytic metaphysics and philosophy of science, which defines the stronger and weaker senses in which the activities of sensory systems may be constitutive for flavour perception. Second, relying on empirical results in flavour science, the constitutive status of activities related to distinct sensory systems in the context of flavour perception is investigated. (shrink)

This is an anthology of new papers by top researchers in epistemology and philosophy of mind focused on the epistemology of non-visual perception. The focus of the volume is to highlight the many different domains in which non-visual sensory experience, broadly construed to include multimodal experience associated with emotional and agential perception, plays a rational role, for instance, as an immediate justifier of belief. -/- .

This collection of essays brings together research on sense modalities in general and spatial perception in particular in a systematic and interdisciplinary way. It updates a long-standing philosophical fascination with this topic by incorporating theoretical and empirical research from cognitive science, neuroscience, and psychology. The book is divided thematically to cover a wide range of established and emerging issues. Part I covers notions of objectivity and subjectivity in spatial perception and thinking. Part II focuses on the canonical distal (...) senses, such as vision and audition. Part III concerns the chemical senses, including olfaction and gustation. Part IV discusses bodily awareness, peripersonal space, and touch. Finally, the volume concludes with Part V, on multimodality. Spatial Senses is an important contribution to the scholarly literature on the philosophy of perception that takes into account important advances in the sciences. (shrink)

Philosophy of perception is one of the most robust and flourishing fields within contemporary philosophy. This volume casts a spotlight on the subject’s most dynamic conversations, with pairs of philosophers debating six key questions: 1. Are perceptual states representations? 2. How can perception be unconscious? 3. Is attention necessary for perception? 4. In what sense is perception multimodal? 5. How is vision different from other sense modalities? 6. Does perception include both low-level and high-level (...) properties? Preliminary descriptions of each side in each debate, annotated bibliographies for each question, and a supplemental guide to further controversies in the field are included. These student-friendly features make the volume is_ _an ideal resource for any advanced student wishing to understand live debates in this active field. (shrink)

Recent evidence in cognitive neuroscience indicates that the visual system is influenced by the outcome of an early appraisal mechanism that automatically evaluates what is seen as being harmful or beneficial for the organism. This indicates that there could be valence in perception. But what could it mean for one to see something positively or negatively? Although most theories of emotions accept that valence involves being related to values, the nature of this relation remains highly debated. Some explain valence (...) in terms of evaluative content, others in terms of evaluative attitude. Here I shall argue that an account of affective perception in terms of attitude has more chance of succeeding. To do so, I will first highlight the difficulties that a content-based approach faces, considering the many forms that it might take. I will conclude that seeing the world positively or negatively involves more than a positive or negative content; it involves a distinctive attitude, but which one? Should it be conceived of in imperative or evaluative terms? And what makes this attitude distinct from a proper emotion? (shrink)

This chapter examines how our sense modalities interact in the perception of persistence. The chapter concentrates on two questions. The first concerns perceptual processing—do perceptual computations of object persistence ever integrate and compute over representations from more than one modality? It argues that this question should be answered affirmatively. The second question concerns perceptual experience—do experiences of object persistence ever exhibit a constitutively multisensory phenomenal character, or is the phenomenology of object persistence always uniquely associated with just one modality? (...) The chapter argues that the available evidence underdetermines the answer to this question, but suggests ways it might be empirically resolved. (shrink)