Sound symbolism refers to non‐arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as “maluma” and “takete” with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity analysis (RSA). (...) Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics‐to‐noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound‐to‐shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross‐modal correspondence as a basis for language users' sensitivity to this type of sound symbolism. (shrink)

Although the target article provides strong evidence against the locationist view, evidence for the constructionist view is inconclusive, because co-activation of brain regions does not necessarily imply connectivity between them. We propose a rigorous approach wherein connectivity between co-activated regions is first modeled using exploratory Granger causality, and then confirmed using dynamic causal modeling or Bayesian modeling.

Synesthetes can be distinguished from nonsynesthetes on a variety of experimental tasks because their concurrent synesthetic experiences can affect task performance if these experiences match or conflict with some aspect of the stimulus. Here, we tested grapheme‐color synesthetes and nonsynesthetic control participants using a novel perceptual similarity task to assess whether synesthetes’ concurrent color experiences influence perceived grapheme similarity. Participants iteratively arranged graphemes and, separately, their associated synesthetic colors in a display, such that similar items were placed close together and (...) dissimilar items further apart. The resulting relative inter‐item distances were used to calculate the pair‐wise (dis)similarity between items in the set, and thence to create separate perceptual representational dissimilarity matrices (RDMs) for graphemes and colors, on an individual basis. On the assumption that synesthetes’ similarity judgments for graphemes would be influenced by their concurrent color experiences, we predicted that grapheme and color RDMs would be more strongly correlated for synesthetes than nonsynesthetes. We found that the mean grapheme‐color RDM correlation was indeed significantly higher in synesthetes than nonsynesthetes; in addition, synesthetes’ grapheme‐color RDM correlations were more likely to be individually statistically significant, even after correction for multiple tests, than those of nonsynesthetes. Importantly, synesthetes’ grapheme‐color RDM correlations scaled with the consistency of their grapheme‐color associations as measured by their Synesthesia Battery (SB) scores. By contrast, the relationship between SB scores and grapheme‐color RDM correlations for nonsynesthetes was not significant. Thus, dissimilarity analysis quantitatively distinguished synesthetes from nonsynesthetes, in a way that meaningfully reflects a key aspect of synesthetic experience. (shrink)

We review evidence for multifaceted functional specialization of somatosensory information processing, both within and outside classical somatosensory cortex. We argue that the nature of such specialization has not yet been clarified adequately to regard the proposed action/perception dichotomy as being established. However, we believe this is a good working hypothesis that can motivate further work.

Grush's emulation theory comprises both modality-specific and amodal emulators. I suggest that the amodal variety be replaced by multisensory emulators. The key distinction is that multisensory processing retains the characteristics of individual sensory modalities, in contrast to amodal processing. The latter term is better reserved for conceptual and linguistic systems, rather than perception or emulation.