The last couple of years have seen a rapid growth of interest in the study of crossmodal correspondences – the tendency for our brains to preferentially associate certain features or dimensions of stimuli across the senses. By now, robust empirical evidence supports the existence of numerous crossmodal correspondences, affecting people’s performance across a wide range of psychological tasks – in everything from the redundant target effect paradigm through to studies of the Implicit Association Test, and from speeded discrimination/classification tasks through (...) to unspeeded spatial localisation and temporal order judgment tasks. However, one question that has yet to receive a satisfactory answer is whether crossmodal correspondences automatically affect people’s performance , as opposed to reflecting more of a strategic, or top-down, phenomenon. Here, we review the latest research on the topic of crossmodal correspondences to have addressed this issue. We argue that answering the question will require researchers to be more precise in terms of defining what exactly automaticity entails. Furthermore, one’s answer to the automaticity question may also hinge on the answer to a second question: Namely, whether crossmodal correspondences are all ‘of a kind’, or whether instead there may be several different kinds of crossmodal mapping . Different answers to the automaticity question may then be revealed depending on the type of correspondence under consideration. We make a number of suggestions for future research that might help to determine just how automatic crossmodal correspondences really are. (shrink)

Cognitive Science, Volume 46, Issue 2, February 2022.

Height-pitch associations are claimed to be universal and independent of language, but this claim remains controversial. The present study sheds new light on this debate with a multimodal analysis of individual sound and melody descriptions obtained in an interactive communication paradigm with speakers of Dutch and Farsi. The findings reveal that, in contrast to Dutch speakers, Farsi speakers do not use a height-pitch metaphor consistently in speech. Both Dutch and Farsi speakers’ co-speech gestures did reveal a mapping of higher pitches (...) to higher space and lower pitches to lower space, and this gesture space-pitch mapping tended to co-occur with corresponding spatial words (high-low). However, this mapping was much weaker in Farsi speakers than Dutch speakers. This suggests that cross-linguistic differences shape the conceptualization of pitch and further calls into question the universality of height-pitch associations. (shrink)

Although acoustic frequency is not a spatial property of physical objects, in common language, pitch, i.e., the psychological correlated of frequency, is often labeled spatially (i.e., “high in pitch” or “low in pitch”). Pitch-height is known to modulate (and interact with) the response of participants when they are asked to judge spatial properties of non-auditory stimuli (e.g., visual) in a variety of behavioral tasks. In the current study we investigated whether the modulatory action of pitch-height extended to the haptic estimation (...) of height of a virtual step. We implemented a HW/SW setup which is able to render virtual 3D objects (stair-steps) haptically through a PHANTOM device, and to provide real-time continuous auditory feedback depending on the user interaction with the object. The haptic exploration was associated with a sinusoidal tone whose pitch varied as a function of the interaction point’s height within (i) a narrower and (ii) a wider pitch range, or (iii) a random pitch variation acting as a control audio condition. Explorations were also performed with no sound (haptic only). Participants were instructed to explore the virtual step freely, and to communicate height estimation by opening their thumb and index finger to mimic the step riser height, or verbally by reporting the height in centimeters of the step riser. We analyzed the role of musical expertise by dividing participants into non musicians and musicians. Results showed no effects of musical pitch on high-realistic haptic feedback. Overall there is no difference between the two groups in the proposed multimodal conditions. Additionally, we observed a different haptic response distribution between musicians and non musicians when estimations of the auditory conditions are matched with estimations in the no sound condition. (shrink)

We assessed the automaticity of spatial-numerical and spatial-musical associations by testing their intentionality and load sensitivity in a dual-task paradigm. In separate sessions, 16 healthy adults performed magnitude and pitch comparisons on sung numbers with variable pitch. Stimuli and response alternatives were identical, but the relevant stimulus attribute (pitch or number) differed between tasks. Concomitant tasks required retention of either color or location information. Results show that spatial associations of both magnitude and pitch are load sensitive and that the spatial (...) association for pitch is more powerful than that for magnitude. These findings argue against the automaticity of spatial mappings in either stimulus dimension. (shrink)

This study tested to what extent young listeners metaphorically conceptualize basic musical relations. Ninety children aged 11 (30 attending a music school and 30 Serbian and 30 Romani children with no musical education) were played 5 stimuli with mutually opposed musical elements and asked to respond what the first and what the second one was like. Their answers were classified into metaphors according to the tenets of the conceptual metaphor theory. The results suggest an overwhelming dominance of metaphorical replies, in (...) which most mappings were based on image schemas within a predominantly visual–spatial modality. There were some differences in conceptualizations, however, with “high and low” tones also perceived as “big and small,” or “upward” musical motion seen as “forward.” This article analyzes the possibility of finding a common denominator for seemingly disparate replies, where the study of musical conceptualization might be instrumental in the cognitive semantic quest for metaphorical universals. (shrink)

The apparent dichotomy between qualitative versus quantitative dimensions in science intersects with the domain of several disciplines, as well as different research fields within one and the same discipline. The perception of qualitative as “poor quantitative,” however, is methodologically unsustainable, because there are perfectly rigorous ways to conduct qualitative research. A somehow different question is whether a science of qualities per se is possible: that is, whether a science of appearances can be devised, what its observables are, and its methodological (...) implications. The article deals with this last issue, and especially from the point of view of experimental phenomenology. The weakness of the current approach to perception is discussed, and an alternative view that goes beyond veridicalism is presented. The analysis tackles crucial issues such as color and spatial appearances, the qualitative dimension of perceived depth, and the role of visual meaningful gestalten as information tools for the survival of living beings in the environment. If developed, a science of the qualitative dimensions of appearances may contribute to revising the ecological theory of perception by making its natural semantics more explicit. (shrink)

Mental body-representations are highly plastic and can be modified after brief exposure to unexpected sensory feedback. While the role of vision, touch and proprioception in shaping body-representations has been highlighted by many studies, the auditory influences on mental body-representations remain poorly understood. Changes in body-representations by the manipulation of natural sounds produced when one's body impacts on surfaces have recently been evidenced. But will these changes also occur with non-naturalistic sounds, which provide no information about the impact produced by or (...) on the body? Drawing on the well-documented capacity of dynamic changes in pitch to elicit impressions of motion along the vertical plane and of changes in object size, we asked participants to pull on their right index fingertip with their left hand while they were presented with brief sounds of rising, falling or constant pitches, and in the absence of visual information of their hands. Results show an "auditory Pinocchio" effect, with participants feeling and estimating their finger to be longer after the rising pitch condition. These results provide the first evidence that sounds that are not indicative of veridical movement, such as non-naturalistic sounds, can induce a Pinocchio-like change in body-representation when arbitrarily paired with a bodily action. (shrink)