As most work on flower foraging focuses on bees, studying Lepidoptera can offer fresh perspectives on how sensory capabilities shape the interaction between flowers and insects. Through a combination of innate preferences and learning, many Lepidoptera persistently visit particular flower species. Butterflies tend to rely on their highly developed sense of colour to locate rewarding flowers, while moths have evolved sophisticated olfactory systems towards the same end. However, these modalities can interact in complex ways; for instance, butterflies’ colour preference (...) can shift depending on olfactory context. The mechanisms by which such cross‐modal interaction occurs are poorly understood, but the mushroom bodies appear to play a central role. Because of the diversity seen within Lepidoptera in terms of their sensory capabilities and the nature of their relationships with flowers, they represent a fruitful avenue for comparative studies to shed light on the co‐evolution of flowers and flower‐visiting insects. (shrink)

Following on from ecological theories of perception, such as the one proposed by [Gibson, J. J. . The senses considered as perceptual systems. Boston: Houghton Mifflin] this paper reviews the literature on the multisensory interactions underlying the perception of flavor in order to determine the extent to which it is really appropriate to consider flavor perception as a distinct perceptual system. We propose that the multisensory perception of flavor may be indicative of the fact that the taxonomy currently (...) used to define our senses is simply not appropriate. According to the view outlined here, the act of eating allows the different qualities of foodstuffs to be combined into unified percepts; and flavor can be used as a term to describe the combination of tastes, smells, trigeminal, and tactile sensations as well as the visual and auditory cues, that we perceive when tasting food. (shrink)

Visuo-motor adaptation to optical prisms displacing the visual scene (prism adaptation, PA) is a method used for investigating visuomotor plasticity in healthy individuals and, in clinical settings, for the rehabilitation of unilateral spatial neglect. In the standard paradigm, the adaptation phase involves repeated pointings to visual targets, while wearing optical prisms displacing the visual scene laterally. Here we explored differences in PA, and its aftereffects (AEs), as related to the sensory modality of the target. Visual, auditory, and multisensory - (...) audio-visual - targets in the adaptation phase were used, while participants wore prisms displacing the visual field rightwards by 10◦. Proprioceptive, visual, visual-proprioceptive, auditory-proprioceptive straight-ahead shifts were measured. Pointing to auditory and to audio-visual targets in the adaptation phase produces proprioceptive, visual-proprioceptive, and auditory-proprioceptive AEs, as the typical visual targets did. This finding reveals that cross-modal plasticity effects involve both the auditory and the visual modality, and their interactions (Experiment 1). Even a shortened PA phase, requiring only 24 pointings to visual and audio-visual targets (Experiment 2), is sufficient to bring about AEs, as compared to the standard 92-pointings procedure. Finally, pointings to auditory targets cause AEs, although PA with a reduced number of pointings (24) to auditory targets brings about smaller AEs, as compared to the 92-pointings procedure (Experiment 3). Together, results from the three experiments extend to the auditory modality the sensorimotor plasticity underlying the typical AEs produced by PA to visual targets. Importantly, PA to auditory targets appears characterized by less accurate pointings and error correction, suggesting that the auditory component of the PA process may be less central to the building up of the AEs, than the sensorimotor pointing activity per se. These findings highlight both the effectiveness of a reduced number of pointings for bringing about AEs, and the possibility of inducing PA with auditory targets, which may be used as a compensatory route in patients with visual deficits. (shrink)

The Bayesian model of multisensory cue integration proposed by Ernst and Banks provides an attractive model for understanding a way that our sensory systems may interact. Moreover, it has been suggested that the process of multisensory integration that it models underpins conscious experiences with multisensory representational contents merged across modalities. Should we therefore take empirical support for the Bayesian model as evidence of the multimodality of perception? Focusing on evidence of integration across vision and touch, I argue (...) that apparent support for the model does not warrant the rejection of the view that each of our conscious perceptual experiences is associated with one and only one sense modality. (shrink)

This chapter examines how our sense modalities interact in the perception of persistence. The chapter concentrates on two questions. The first concerns perceptual processing—do perceptual computations of object persistence ever integrate and compute over representations from more than one modality? It argues that this question should be answered affirmatively. The second question concerns perceptual experience—do experiences of object persistence ever exhibit a constitutively multisensory phenomenal character, or is the phenomenology of object persistence always uniquely associated with just one modality? (...) The chapter argues that the available evidence underdetermines the answer to this question, but suggests ways it might be empirically resolved. (shrink)

Cognitive scientists have long known that the modalities interact during perceptual processing. Cross-modal illusions like the ventriloquism effect show that the course of processing in one modality can alter the course of processing in another. But how do the modalities interact in the specific domain of object perception? This paper distinguishes and analyzes two kinds of multisensory interaction in object perception. First, the modalities may bind features to a single object or event. Second, the modalities may cooperate when (...) differentiating an object or event from its surroundings. I critically evaluate evidence for various forms of multisensory binding. I then consider the case for multisensory differentiation. I argue that existing evidence for multisensory differentiation is inconclusive. I highlight ways that the issue might be empirically resolved. (shrink)

In the social world, multiple sensory channels are used concurrently to facilitate communication. Among human and nonhuman pri- mates, faces and voices are the primary means of transmitting social signals (Adolphs, 2003; Ghazanfar and Santos, 2004). Primates recognize the correspondence between species-specific facial and vocal expressions (Massaro, 1998; Ghazanfar and Logothetis, 2003; Izumi and Kojima, 2004), and these visual and auditory channels can be integrated into unified percepts to enhance detection and discrimination. Where and how such communication signals are integrated (...) at the neural level are poorly understood. In particular, it is unclear what role “unimodal” sensory areas, such as the auditory cortex, may play. We recorded local field potential activity, the signal that best correlates with human imaging and event-related potential signals, in both the core and lateral belt regions of the auditory cortex in awake behaving rhesus monkeys while they viewed vocalizing conspecifics. We demonstrate unequivocally that the primate auditory cortex integrates facial and vocal signals through enhancement and suppression of field potentials in both the core and lateral belt regions. The majority of these multisensory responses were specific to face/voice integration, and the lateral belt region shows a greater frequency of multisensory integration than the core region. These multisensory processes in the auditory cortex likely occur via recip- rocal interactions with the superior temporal sulcus. (shrink)

Summary In natural face-to-face interactions, verbal communication always occurs in association with expressions of nonverbal behavior. The functional contribution of these multimodal aspects to the meaning of the message and to its effects fulfils multiple communicative functions that differ according primarily to the speaker’s intentions, to the interpersonal relations between the speaker and the addressee, to the nature of the message, and to the context. When nonverbal behavior is reproduced in a written literary text, it becomes functional to the textual (...) and narrative process as it serves as a signifier for the reader. A fictional character is never fixed and unchanging. Through writing, each author encourages the explicit or implicit evocation of a multisensory world, which readers decode and reconstruct, inevitably conditioned by their cognitive and cultural environment. In this paper, we refer to Salinger’s famous novel The Catcher in the Rye to analyze the literary valence of representing the characters’ multisensory communication, focusing on the core relationship between the explicit and the implicit parts in reconstructing the psychological depth of a literary character. (shrink)

Linking arbitrary shapes to personal labels or reward values results in immediate processing benefits for those stimuli that happen to be associated with the self or high rewards in perceptual matching tasks. Here we further explored how social and reward associations interact with multisensory stimuli by pairing labels and objects with tones. We also investigated whether self and reward biases persist for multisensory stimuli with the label removed after an association had been made. Both high reward stimuli and (...) those associated with the self, resulted in faster responses and improved discriminability, which persisted for multisensory stimuli even when the labels were removed. However, these self- and reward-biases partly depended on the specific alignment between the physical tones and the conceptual order. Performance for reward associations improved when the endpoints of low or high rewards were paired with low or high tones; meanwhile, for personal associations, there was a benefit when the self was paired with either low or high tones, but there was no effect when the stranger was associated with either endpoint. These results indicate that, unlike reward, social personal associations are not represented along a continuum with two marked endpoints but rather with a single reference point. (shrink)

The reach-to-grasp movement is ordinarily performed in everyday living activities and it represents a key behavior that allows humans to interact with their environment. Remarkably, it serves as an experimental test case for probing the multisensory architecture of goal-oriented actions. This review focuses on experimental evidence that enhances or modifies how we might conceptualize the “multisensory” substrates of prehension. We will review evidence suggesting that how reach-to-grasp movements are planned and executed is influenced by information coming from different (...) sensory modalities such as vision, proprioception, audition, taste, and olfaction. The review closes with some considerations about the predominant role of the multisensory constituents in shaping prehensile behavior and how this might be important for future research developments, especially in the rehabilitative domain. (shrink)

This paper concerns epistemic developments in the field of sensory perception. I argue that Uexküll’s concept of the Umwelträume and certain principles of multisensory integration explain and describe in similar terms the manner in which different sensory modalities interact. Indeed, they both concern knowledge, describing in spatial terms how the mind makes itself up, makes up its objects, and how the objects, in turn, make up the mind. My intention is to set side by side these two trends of (...) thought in order to mutually explain them. I suggest that there is some sort of dialectic at work between the Umwelträume and multisensory integration: in the first case interaction between the senses results in the creation of high definition subjects, whereas in the latter the interaction of the senses results in high definition objects. (shrink)

The reality-virtuality continuum encompasses a multitude of objects, events and environments ranging from real-world multisensory inputs to interactive multisensory virtual simulators, in which sensory integration can involve very different combinations of both physical and digital inputs. These different ways of stimulating the senses can affect the consumer’s consciousness, potentially altering their judgements and behaviours. In this perspective paper, we explore how technologies such as Augmented Reality and Virtual Reality can, by generating and modifying the human sensorium, act on (...) consumer consciousness. We discuss the potential impact of this altered consciousness for consumer behaviour while, at the same time, considering how it may pave the way for further research. (shrink)

Philosophical considerations as well as several recent studies from neurophysiology, neuropsychology, and psychophysics converged in showing that the peripersonal space is structured in a body-centred manner and represented through integrated sensory inputs. Multisensory representations may deserve the function of coding peripersonal space for avoiding or interacting with objects. Neuropsychological evidence is reviewed for dynamic interactions between space representations and action execution, as revealed by the behavioural effects that the use of a tool, as a physical extension of the reachable (...) space, produces on visual–tactile extinction. In particular, tool-use transiently modifies action space representation in a functionally effective way. The possibility is discussed that the investigation of multisensory space representations for action provides an empirical way to consider in its specificity pre-reflexive self-consciousness by considering the intertwining of self-relatedness and object-directness of spatial experience shaped by multisensory and sensorimotor integrations. (shrink)

Psychophysics and neuroscience demonstrate that different sensory systems interact and influence each other. Perceiving involves extensive cooperation and coordination among systems associated with sight, hearing, touch, smell, and taste. Nonetheless, it remains unclear in what respects conscious perceptual awareness is multisensory. This paper distinguishes six differing varieties of multisensory awareness, explicates their consequences, and thereby elucidates the multisensory nature of perception. It argues on these grounds that perceptual awareness need not be exhausted by that which is associated (...) with each of the respective sensory modalities along with whatever accrues thanks to simple co-consciousness. (shrink)

It is common to distinguish between “holist” and “reductionist” views of brain function, where the former envisions the brain as functioning as an indivisible unit and the latter as a collection of distinct units that serve different functions. Opposing reductionism, a number of researchers have pointed out that cortical network architecture does not respect functional boundaries, and the neuroanatomist V. Braitenberg proposed to understand the cerebral cortex as a “great mixing machine” of neuronal activity from sensory inputs, motor commands, and (...) intrinsically generated processes. In this paper, we offer a contextualization of Braitenberg’s point, and we review evidence for the interactions of neuronal activity from multiple sensory inputs and intrinsic neuronal processes in the cerebral cortex. We focus on new insights from studies on audiovisual interactions and on the influence of respiration on brain functions, which do not seem to align well with “reductionist” views of areal functional boundaries. Instead, they indicate that functional boundaries are fuzzy and context dependent. In addition, we discuss the relevance of the influence of sensory, proprioceptive, and interoceptive signals on cortical activity for understanding brain-body interactions, highlight some of the consequences of these new insights for debates on embodied cognition, and offer some suggestions for future studies. (shrink)

The majority of the world’s population now lives an urban existence, spending as much as 95% of their lives indoors. The olfactory atmosphere in the built environment has been shown to exert a profound, if often unrecognized, influence over our mood and well-being. While the traditionally malodorous stench to be found indoors (i.e., prior to the invention of modern sanitation) has largely been eliminated in recent centuries, many of the outbreaks of sick-building syndrome that have been reported over the last (...) half century have been linked to the presence of a strange smell in the environment. At the same time, however, there is also growing evidence that consumer behavior can be manipulated by the presence of pleasant ambient odors, while various aromatherapy scents are said to improve our mood and well-being. This Anglophone review focuses primarily on indoor western urban developed spaces. Importantly, the olfactory ambience constitutes but one component of the multisensory atmosphere and ambient odors interact with the visual, auditory, and haptic aspects of the built environment. Surprisingly, the majority of published studies that have deliberately chosen to combine ambient scent with other sensory interventions, such as, for example, music, have failed to increase store sales, or to enhance people’s mood and/or well-being, as might have been expected. Such negative findings therefore stress the importance of considering multisensory congruency while, at the same time, also highlighting the potential dangers that may be associated with sensory overload when thinking about the effect of ambient smell on our well-being. (shrink)

The last decade has seen great progress in the study of the nature of crossmodal links in exogenous and endogenous spatial attention . Exogenous spatial cuing studies of human crossmodal attention and multisensory integration. In C. Spence, & J. Driver , Crossmodal space and crossmodal attention . Oxford, UK: Oxford University Press.], for a recent review). A growing body of research now highlights the existence of robust crossmodal links between auditory, visual, and tactile spatial attention. However, until recently, studies (...) of exogenous and endogenous attention have proceeded relatively independently. In daily life, however, these two forms of attentional orienting continuously compete for the control of our attentional resources, and ultimately, our awareness. It is therefore critical to try and understand how exogenous and endogenous attention interact in both the unimodal context of the laboratory and the multisensory contexts that are more representative of everyday life. To date, progress in understanding the interaction between these two forms of orienting has primarily come from unimodal studies of visual attention. We therefore start by summarizing what has been learned from this large body of empirical research, before going on to review more recent studies that have started to investigate the interaction between endogenous and exogenous orienting in a multisensory setting. We also discuss the evidence suggesting that exogenous spatial orienting is not truly automatic, at least when assessed in a crossmodal context. Several possible models describing the interaction between endogenous and exogenous orienting are outlined and then evaluated in terms of the extant data. (shrink)

This book offers an overview of the emerging SID research, discussing theories, methods, and practices, with a focus on the multisensory aspects of sonic experience.

Multisensory integration allows us to link sensory cues from multiple sources and plays a crucial role in speech development. However, it is not clear whether humans have an innate ability or whether repeated sensory input while the brain is maturing leads to efficient integration of sensory information in speech. We investigated the integration of auditory and somatosensory information in speech processing in a bimodal perceptual task in 15 young adults (age 19 to 30) and 14 children (age 5 to (...) 6). The participants were asked to identify if the perceived target was the sound /e/ or /ø/. Half of the stimuli were presented under a unimodal condition with only auditory input. The other stimuli were presented under a bimodal condition with both auditory input and somatosensory input consisting of facial skin stretches provided by a robotic device, which mimics the articulation of the vowel /e/. The results indicate that the effect of somatosensory information on sound categorization was larger in adults than in children. This suggests that integration of auditory and somatosensory information evolves throughout the course of development. (shrink)

I argue for sensory pluralism. This is the view that there are many forms of sensory interaction and unity, and no single category that classifies them all. In other words, sensory interactions do not form a single natural kind. This view suggests that how we classify sensory systems (and the experiences they generate) partly depends on our explanatory purposes. I begin with a detailed discussion of the issue as it arises for our understanding of thermal perception, followed by a (...) general account and defense of sensory pluralism. (shrink)

Color conveys critical information about the flavor of food and drink by providing clues as to edibility, flavor identity, and flavor intensity. Despite the fact that more than 100 published papers have investigated the influence of color on flavor perception in humans, surprisingly little research has considered how cognitive and contextual constraints may mediate color–flavor interactions. In this review, we argue that the discrepancies demonstrated in previously-published color–flavor studies may, at least in part, reflect differences in the sensory expectations that (...) different people generate as a result of their prior associative experiences. We propose that color–flavor interactions in flavor perception cannot be understood solely in terms of the principles of multisensory integration but that the role of higher-level cognitive factors, such as expectations, must also be considered. (shrink)

With the advancement of digital technology in contemporary art, new hybrid forms of interaction emerge that invite viewers to make images present in physical space as events that claim a life of their own. In breaking away from representational and performance art theories that have dominated the critique of new media artwork since the 1980s, this article analyses an iconic vision of mobile touchscreens based on the medieval Byzantine chorographic inscription of the sacred in profane spaces. As defined in (...) recent art historical studies on Byzantine icons, chorography builds on a multisensory spatial interaction between the beholders of icons that results in feeling the presence of a divine, invisible image. In light of postmodern critique of digital images’ capacity to manipulate notions of reality, new media aesthetic theory hardly addresses this Byzantine iconic vision that is fundamental to western visual culture. Jeffrey Shaw’s installation, The Golden Calf, is discussed to offer an alternative in understanding how digital and physical spaces function together to evoke something essentially real. (shrink)

Sensorial access to products in shop encounters constitutes a crucial aspect of the appeal to customers. This paper examines sensorial engagements with products in a specific ecology with a focus on the possible opening of a shop encounter. When passers-by stroll from one stand to another, open to local findings, unplanned discoveries, and emergent opportunities to buy, they orient to the sensory appeal of the products, becoming possible customers, stopping in front of a counter and engaging in a social (...) class='Hi'>interaction with sellers and other customers. The analysis focuses on multisensoriality in action, studying how customers and sellers treat the sensorial qualities of the products, the relevance of the sensory engagement of the customer with their materiality, and their involvement in the social encounter with the seller, including normative expectations related to sensing and buying. It includes a discussion of how under Covid-19 sensorial access to products was restricted and the sensorial, interactional, normative, and economic consequences of these restrictions. Based on video recordings of market encounters, and on an ethnomethodological and conversation analytic approach of multimodality and multisensoriality, the paper reflects on how appealing to the senses of sight, touch, smell, and taste is methodically orchestrated, and its consequences for the interaction with the seller, the sensorial experience, and the economic transaction. (shrink)

Synaesthesia is a strange sensory blending: synaesthetes report experiences of colours or tastes associated with particular sounds or words. This volume presents new essays by scientists and philosophers exploring what such cases can tell us about the nature of perception and its boundaries with illusion and imagination.

Haptic touch is an inherently active and exploratory form of perception, involving both coordinated movements and an array of distinct sensory receptors in the skin. For this reason, some have claimed that haptic touch is not a single sense, but rather a multisensory collection of distinct sensory systems. Though this claim is often made, it relies on what I regard as a confused conception of multisensory interaction. In its place, I develop a nuanced hierarchy of multisensory (...) involvement. According to this hierarchy, touch turns out to be a single modality in that its various receptors assign their features to the same tangible objects. When we grasp an object a range of distinct properties?shape, warmth, heft, texture, etc.?are all felt to belong to the object, just as different visual properties are associated with a visual object. Paradigm multisensory experiences, on the other hand, involve associations between distinct perceptual experiences, as when the way something looks affects the way something sounds. Thus despite its functional and physiological diversity, haptic touch can be regarded as a single sense. (shrink)

The increasing use of virtual laboratories in education raises new philosophical—and perhaps especially phenomenological—questions related to how this type of technological mediation affects the user’s sense of situated embodied being: sensory perception. The empirical basis of this phenomenological inquiry is a case study conducted in a Danish school setting. This allows us to compare analog laboratory work with virtual. Inspired by Maurice Merleau-Ponty, we describe how pupils’ bodily and multisensory interactions with laboratory tools differ across physical and virtual settings. (...) Virtual laboratories are complex, sociotechnical, often opaque practices that affect the pupils’ sense of embodiment, thus prompting the need for in situ development of hermeneutical strategies for bridging the gap between the simulated laboratory and the physical world. In the final section, we discuss how these strategies can be considered posthuman learning processes. (shrink)

Philosophers and scientists have studied sensory perception and, in particular, vision for many years. Increasingly, however, they have become interested in the nonvisual senses in greater detail and the problem of individuating the senses in a more general way. The Aristotelian view is that there are only five external senses—smell, taste, hearing, touch, and vision. This has, by many counts, been extended to include internal senses, such as balance, proprioception, and kinesthesis; pain; and potentially other human and nonhuman senses. This (...) “multisensory turn” has been driven partly by developments in contemporary psychology and neuroscience, which have revealed a host of complex interrelations and interactions between sensory modalities previously thought to be distinct. Contrasts between modalities and other crossmodal phenomena, including multisensory integration, synesthesia, and sensory substitution, have also begun to receive more attention in a burgeoning scientific and philosophical literature on multisensory perception and other crossmodal effects. This article focuses on recent empirically informed contributions to the philosophy of perception, as well as key scientific works that provide important background information and insights into the nature of the senses and sensory perception. Indeed, one of the lessons of the multisensory turn, and of contemporary philosophy of mind more generally, is that philosophers ignore this body of empirical research at their peril because many human and animal senses turn out to be richer and more complex than philosophers and scientists had previously imagined, making this a fruitful area for interdisciplinary interaction and research. (shrink)

In the age of ubiquitous technology, humans are reshaped through each transaction they are involved in. AI-driven networks, online games, and multisensory interactive environments make up alternate realities. Within such alternate worlds, users are reshaped as deterministic agents. Technology’s focus on reducing complexity leads to a human being dependent on prediction-driven machines and behaving like them. Meaning and information are disconnected. Existence is reduced to energy processes. The immense gain in efficiency translates as prosperity. Citizens of advanced economies, hurrying (...) in the rhythm of machine-driven interactions, feel entitled to it. Successful at the price of self-awareness, they no longer know what this means. Happiness and prosperity are not consubstantial. Lack of happiness leads to aggression. This is the image of the world as we see it, no longer looking at each other, eye to eye, but screen to screen. The questions eliminated in the process of transferring responsibility from the individual to machines will inevitably become society’s new focus. When the goal is to get everyone to behave like a machine, the Singularity hypothesis becomes self-fulfilling prophecy. If, in addition to having exhausted natural resources, society does not want to end up making the human mind superfluous, it has to seek a better understanding of what makes anticipation possible. As a definitory characteristic of the living, corresponding to its complexity, anticipation can no longer be taken for granted, while every effort is made to reduce complexity for efficiency’s sake. Awareness of the processes conducive to its expression in successful human action will position human beings as masters of their destiny, not slaves of their own making. Antecapere ergo sum might be the counterclaim to Descartes’ Dubito ergo sum . To resist being perfected into oblivion, that is, unsustainable prosperity, means to reclaim the knowledge corresponding to higher levels of complexity. (shrink)

Higher animals need to identify and track material objects because they depend on interactions with them for nutrition, reproduction, and social interaction. This paper investigates the perception of material objects. It argues, first, that material objects are tagged, in all five external senses, as bearers of the features detected by them. This happens through a perceptual process, here entitled Generalized Completion, which creates the appearance of objects that have properties that transcend the activation of sensory receptors. The paper shows, (...) secondly, that material objects are privileged subjects for perceived motion and interaction. That is, they are perceived as subjects for these properties while their parts seem to be subjects only derivatively. Material objects are the only perceptual subjects that are both multisensory and privileged. (shrink)

From an architectural perspective, the purpose of this paper is to explore the dynamic relations between individuals and the environments using the concept of affordance. In three different cases of architectural research, the concept of affordance is used as an analytical tool – yet demonstrating different scopes and outcomes. A post-occupancy evaluation of an office space in transformation; a lighting assessment and intervention in low vision rehabilitation situated in private home environments; and an urban event at an architecture festival involving (...) the intervention of a sound- and scentscape. The three cases show that the concept of affordance can embrace and operationalize the human perception of the environment, pointing at descriptive, prescriptive, and exploratory possibilities in relation to architectural design and research. The concept of affordance enables rich descriptions and explorations of qualities of architecture, related to multisensorial, emotional, and social interactional aspects of humans and their environments. (shrink)

Interactions in service ecosystems, as opposed to the service dyad, have recently gained much attention from research. However, it is still unclear how they influence a customer’s experiential value and trigger desired prosocial behavior. The purpose of this study is to identify which elements of the multi-actor service ecosystem contribute to a customer’s experiential value and to investigate its relation to a customer’s interaction attitude and inter-customer helping behavior. The authors adopted a scale development procedure from the existing literature. (...) Service, brand, retail and tourism management research as well as expert feedback is used to generate a pool of 33 items. Exploratory and confirmatory factor analysis were conducted. The scale was validated based on more than 468 responses to a CASI at one of the world’s largest trade shows. Scale-development procedure was followed by structural equation modeling. CFA supports that experiential value in multi-actor ecosystems comprises five dimensions. The functional value of personnel, the perception of other customers’ appearance, the perception of other customers’ behavior, multisensory stimuli, and a customer’s enjoyment. Experiential value positively and directly relates to a customer’s interaction attitude and inter-customer helping behavior. Furthermore, the effect of experiential value on inter-customer helping behavior is partially mediated by interaction attitude. Managers interested in getting more out of interactions with customers will develop an understanding for the interplay between the physical environment and individuals within a multi-actor ecosystem. Social scientists and managers can use the scale to assess experiential value, encourage a customer’s interaction attitude and utilize the customers’ influence on their peers. This paper synthesizes insights from existing research on experiential value, from various fields, in one scale. This holistic approach is the first to simultaneously account for a customer’s interactions with the multisensory physical environment, personal interactions with employees and interactions between customers in a multi-actor service ecosystem. (shrink)

_ Source: _Volume 40, Issue 1, pp 1 - 31 This anthropologically informed study explores descriptions of communication with invisible, superhuman agents in high functioning young adults on the autism spectrum. Based on material from interviews, two hypotheses are formulated. First, autistic individuals may experience communication with bodiless agents as less complex than interaction with peers, since it is unrestricted by multisensory input, such as body language, facial expressions, and intonation. Second, descriptions of how participants absorb into “imaginary (...) realities” suggest that such mental states are desirable due to qualities that facilitate social cognition: While the empirical world comes through as fragmented and incoherent, imaginary worlds offer predictability, emotional coherence, and benevolent minds. These results do not conform to popular expectations that autistic minds are less adapted to experience supernatural agents, and it is instead argued that imaginative, autistic individuals may embrace religious and fictive agents in search for socially and emotionally comprehensible interaction. (shrink)

Audiovisual interaction in speech perception is well defined in adults. Despite the large body of evidence suggesting that children are also sensitive to visual input, very few empirical studies have been conducted. To further investigate whether visual inputs influence auditory perception of phonemes in preschoolers in the same way as in adults, we conducted an audiovisual identification test. The auditory stimuli were presented either in an auditory condition only or simultaneously with a visual presentation of the articulation of the (...) vowel /e/ or /ø/. The results suggest that, although all participants experienced visual influence on auditory perception, substantial individual differences exist in the 5- to 6-year-old group. While additional work is required to confirm this hypothesis, we suggest that auditory and visual systems are developing at that age and that multisensory phonological categorization of the rounding contrast took place only in children whose sensory systems and sensorimotor representations were mature. (shrink)

The sounds that result from our movement and that mark the outcome of our actions typically convey useful information concerning the state of our body and its movement, as well as providing pertinent information about the stimuli with which we are interacting. Here we review the rapidly growing literature investigating the influence of non-veridical auditory cues (i.e., inaccurate in terms of their context, timing, and/or spectral distribution) on multisensory body and action perception, and on motor behavior. Inaccurate auditory cues (...) provide a unique opportunity to study cross-modal processes: the ability to detect the impact of each sense when they provide a slightly different message is greater. Additionally, given that similar cross-modal processes likely occur regardless of the accuracy or inaccuracy of sensory input, studying incongruent interactions are likely to also help us predict interactions between congruent inputs. The available research convincingly demonstrates that perceptions of the body, of movement, and of surface contact features (e.g., roughness) are influenced by the addition of non-veridical auditory cues. Moreover, auditory cues impact both motor behavior and emotional valence, the latter showing that sounds that are highly incongruent with the performed movement induce feelings of unpleasantness (perhaps associated with lower processing fluency). Such findings are relevant to the design of auditory cues associated with product interaction, and the use of auditory cues in sport performance and therapeutic situations given the impact on motor behavior. (shrink)

Noisefest! is an interactive, multisensory experience centered around a small Maine town and rooted in the sounds and noise of its streets. Comprising a Virtual Reality tour, soundwalks and remixes, a 2D laser cut geographical map with Arduino controllers, and a Futuristic noise intoner, one of the objectives of this collaborative, transdisciplinary, and theory-based project is to create concrete opportunities for students to participate in the “real” world and engage with the materiality of noise and its manifestations by interacting (...) with the soundscape through novel, interactive, and multisensory practices. Noisefest! is also an example of how one can creatively and artistically extend the reach of the digital humanities beyond the borders of academia and into the public realm. (shrink)

The recent distinction between sense of agency and sense of body-ownership has attracted considerable empirical and theoretical interest. The respective contributions of central motor signals and peripheral afferent signals to these two varieties of body experience remain unknown. In the present review, we consider the methodological problems encountered in the empirical study of agency and body-ownership, and we then present a series of experiments that study the interplay between motor and sensory information. In particular, we focus on how multisensory (...) signals interact with body representations to generate the sense of body-ownership, and how the sense of agency modulates the sense of body-ownership. Finally, we consider the respective roles of efferent and afferent signals for the experience of one’s own body and actions, in relation to self-recognition and the recognition of other people’s actions. We suggest that the coherent experience of the body depends on the integration of efferent information with afferent information in action contexts. Overall, whereas afferent signals provide the distinctive content of one’s own body experience, efferent signals seem to structure the experience of one’s own body in an integrative and coherent way. (shrink)

This article aims to explore the relation between body and space – specifically how the relation between the embodied awareness of movement and the sense of one’s body-space can be modified and changed deliberately in different kinds of dance practices. Using a multi-sited design, the ethnographical fieldwork, which formed the empirical ground for the study, was from the outset focused on acknowledging the diversity of the dancers’ practices. Each in their own way, the 13 professional dancers involved in the study (...) relate to and experience bodied potentialities, body-space and the spatiality of movement differently. Through their practice, they indicate that the body image is shaped through a multisensorial process of reification, and that seeing is to be related to a broader perceptual engagement. Furthermore, they exemplify how seeing can be deliberately used to expand the sense of their body-space and thereby to affect the spatiality of the fields of their embodied interaction. (shrink)

Philosophy of perception is guilty of focusing on the perception of far space, neglecting the possibility that the perception of the space immediately surrounding the body, which is known as peripersonal space, displays different properties. Peripersonal space is the space in which the world is literally at hand for interaction. It is also the space in which the world can become threatening and dangerous, requiring protective behaviours. Recent research in cognitive neuroscience has yielded a vast array of discoveries on (...) the multisensory and sensorimotor specificities of the processing of peripersonal space. Yet very little has been done on their philosophical implications. Here I will raise the following question: in what manner does the visual experience of a big rock close to my foot differ from the visual experience of the moon in the sky? (shrink)

This chapter critically assesses recent arguments that acquiring the ability to categorize an object as belonging to a certain high-level kind can cause the relevant kind property to be represented in visual phenomenal content. The first two arguments, developed respectively by Susanna Siegel (2010) and Tim Bayne (2009), employ an essentially phenomenological methodology. The third argument, developed by William Fish (2013), by contrast, is supported by an array of psychophysical and neuroscientific findings. I argue that while none of these arguments (...) ultimately proves successful, there is a substantial body of empirical evidence that information originating outside the visual system can nonetheless modulate the way an object’s low-level attributes visually appear. Visual phenomenal content, I show, is not only significantly influenced by crossmodal interactions between vision and other exteroceptive senses such as touch and audition, but also by interactions between vision and non-perceptual systems involved in motor planning and construction of the proprioceptive body-image. (shrink)

This paper introduces a novel perspective on Agentive Cognitive Construction Grammar (AgCCxG) by examining the intricate interplay between mind and language through the lens of both Active Inference and Peircean semiotics. AgCCxG emphasizes the impact of intention and purpose on linguistic choices as a cognitive imperative to balance the symbolic Self (Intelligent Agent) with the dynamics of the environment. Among other things, the paper posits that linguistic constructions, particularly Constructional Attachment Patterns (CAPs), like argument structure constructions, embody experienced interactions with (...) the world through reenactment routines via the integration of multisensory channels. Unlike traditional usage-based approaches (e.g., construction grammars), AgCCxG embraces a robust theory of signs that reveals human representation as a continuous process of semiotic hybridization for the creative prediction of uncertain scenarios. Importantly, the paper challenges the notion of the mind as a unified, rational, uncertainty-reducing machine by asserting that physical processes governing open biological systems profoundly influence the linguistic sign system. Intelligent agents’ adaptability in expressing incongruous realities thus highlights the role of semiotic hybridization in preserving an agent’s autonomy and semiotic boundary. (shrink)

Reading and writing are increasingly performed with digital, screen-based technologies rather than with analogue technologies such as paper and pen. The current digitization is an occasion to “unpack,” theoretically and conceptually, what is entailed in reading and writing as embodied, multisensory processes involving audiovisual and ergonomic interaction with devices having particular affordances. Highlighting the sensorimotor contingencies of substrates and technologies — how movement and object manipulation affect perception, experience, and sensory “feel” — this article presents an embodied approach (...) to reading, writing, and literacy, using three cases of digitization as illustrations of some educational implications: beginning writing by hand or by keyboard; dialogic reading with iPads and print picture books in kindergarten; and deep reading of long, linear texts on paper and on screens. (shrink)

This article proposes a sensory studies methodology for the interpretation of museum objects. The proposed method unfolds in two phases: virtual encounter via an on-line catalog and actual exposure in the context of a handling workshop. In addition to exploring the écart between image and object, the “Sensing Art and Artifacts” exercise articulates a framework for arriving at a multisensory, cross-cultural, interactive understanding of aesthetic value. The case studies presented here involve four objects from the collection of the Hunterian (...) Museum as sensed and interpreted by scholars of psychology, philosophy, sociology, and anthropology. It is proposed that aesthetic judgment in the expanded (cross-cultural) sense contemplated here involves apprehending the museum object through multiple sensory modalities in place of the conventional Western fixation on visible form. (shrink)

The embodiment approach has shown that motor neural networks are involved in the processing of action verbs. There is developmental evidence that embodied effects on verb processing are already present in early years. Yet, the ontogenetic origin of this motor reuse in action verbs remains unknown. This longitudinal study investigates the co-occurrence of manual verbs and actions during mother-child daily routines when children were 1 to 2 and 2 to 3 years old. Eight mother-child dyads were video-recorded in 3-month intervals (...) across 12 months, and the timing of verbs and manual actions were coded by independent observers. Results showed that the probability of matched verb-action co-occurrences were much higher than that of random co-occurrences for Group 1 and Group 2, respectively. The distributions of the verb-action temporal intervals in both groups were quite symmetrical and skewed with the peak corresponding to both 0.00 s synchronic intervals and the shortest +5 s interval. Mother-led instances occurred in both groups whereas child-led instances were restricted to Group 2. Mothers pragmatically aligned their verbal productions, since they repeatedly used those verbs they shared with their children’s repertoire. In conclusion, the early multisensory communicative and manipulative scene affords grounding of verb meanings on the ongoing actions, facilitating verb-action pairing in the realm of social interactions, providing a new dimension to the prevailing solipsistic approach to embodiment. (shrink)