In this paper we offer a theory of cross-modal objects. To begin, we discuss two kinds of linkages between vision and audition. The first is a duality. The the visual system detects and identifies surfaces ; the auditory system detects and identifies sources . Surfaces are illuminated by sources of light; sound is reflected off surfaces. However, the visual system discounts sources and the auditory system discounts surfaces. These and similar considerations lead to the Theory of Indispensable Attributes that states (...) the conditions for the formation of gestalts in the two modalities. The second linkage involves the formation of audiovisual objects, integrated cross-modal experiences. We describe research that reveals the role of cross-modal causality in the formation of such objects. These experiments use the canonical example of a causal link between vision and audition: a visible impact that causes a percussive sound. (shrink)

An audiovisual correspondence (AVC) refers to an observer’s seemingly arbitrary yet consistent matching of sensory features across the two modalities; for example, between an auditory pitch and visual size. Research on AVCs has frequently used a speeded classification procedure in which participants are asked to rapidly classify an image when it is either accompanied by a congruent or an incongruent sound (or vice versa). When, as is typically the case, classification is faster in the presence of a congruent stimulus, researchers (...) have inferred that the AVC is automatic and bottom-up. Such an inference is incomplete because the procedure does not show that the AVC is not subject to top-down influences. To remedy this problem, we devised a procedure that allows us to assess the degree of “bottom-up-ness” and “top-down-ness” in the processing of an AVC. We did this in studies of AVCs between pitch and five visual features: size, height, spatial frequency, brightness, and angularity. We find that all the AVCs we studied involve both bottom-up and top-down processing, thus undermining the prevalent generalization that AVCs are automatic. (shrink)

Shepard has supposed that the mind is stocked with innate knowledge of the world and that this knowledge figures prominently in the way we see the world. According to him, this internal knowledge is the legacy of a process of internalization; a process of natural selection over the evolutionary history of the species. Shepard has developed his proposal most fully in his analysis of the relation between kinematic geometry and the shape of the motion path in apparent motion displays. We (...) argue that Shepard has made a case for applying the principles of kinematic geometry to the perception of motion, but that he has not made the case for injecting these principles into the mind of the percipient. We offer a more modest interpretation of his important findings: that kinematic geometry may be a model of apparent motion. Inasmuch as our recommended interpretation does not lodge geometry in the mind of the percipient, the motivation of positing internalization, a process that moves kinematic geometry into the mind, is obviated. In our conclusion, we suggest that cognitive psychologists, in their embrace of internal mental universals and internalization may have been seduced by the siren call of metaphor. Key Words: apparent motion; imagery; internalization; inverse projection problem; kinematic geometry; measurement; metaphors of mind. (shrink)

La psychologie a toujours traité le comportement et l’expérience comme étant enchâssés dans un flux temporel unidimensionnel, « le courant du comportement » dans lequel les événements et les actions occupent des intervalles de temps qui ne se chevauchent pas. Pourtant, une analyse phénoménologique révèle que la structure de nos vies est bien plus riche et intéressante. En utilisant la notion de « quasidécomposabilité » de Herbert Simon, je décris cette structure comme un assemblage d’épisodes quasi-indépendants se réalisant de façon (...) concurrente, et de nature asynchrone. Il s’agit d’une « structure profonde» des vies contrairement à la conception courante qui conçoit les vies comme « plates » Psychology has always treated behavior and experience as embedded in a unidimensional flow in time, the "stream of behavior." This means that events and actions occupy non-overlapping time-intervals in this stream. Nevertheless a phenomenological analysis reveals that the structure of lives is richer and far more interesting. Using Herb Simon's notion of near-decomposability, I describe the structure of lives as a composite of nearly independent strands that run concurrently, and are asynchronous. This is a "deep structure"' of lives in contrast to the current conception, which conceives of lives as "flat." . (shrink)

We explore experimental methods used to study the phenomena of perceptual organization, first studied by the Gestalt psychologists. We describe an application of traditional psychophysics to perceptual organization and offer alternative methods. Among these, we distinguish two approaches that use multistable stimuli: (1) phenomenological psychophysics, in which the observer's response is assumed to accurately and directly reflect perceptual experience; and (2) the interference paradigm, in which an observer's response is evaluated as correct or incorrect because it pertains to a corrigible (...) task, but does not directly reflect the observer's experience. We show that phenomenological psychophysics can yield valuable information about perceptual organization and lends itself to the development of quantitative theory. We discuss some criticisms of the method and argue that the two approaches that use multistable stimuli are complementary. We also compare each of the approaches with traditional psychophysics. We conclude that the several methods are convergent. (shrink)

Pylyshyn is willing to assume that attention can influence feature integration. We argue that he concedes too much. Feature integration occurs preattentively, except in the case of certain “perverse” displays, such as those used in feature-conjunction searches.

La psychologie a toujours traité le comportement et l’expérience comme étant enchâssés dans un flux temporel unidimensionnel, « le courant du comportement » dans lequel les événements et les actions occupent des intervalles de temps qui ne se chevauchent pas. Pourtant, une analyse phénoménologique révèle que la structure de nos vies est bien plus riche et intéressante. En utilisant la notion de « quasidécomposabilité » de Herbert Simon, je décris cette structure comme un assemblage d’épisodes quasi-indépendants se réalisant de façon (...) concurrente, et de nature asynchrone. Il s’agit d’une « structure profonde» des vies contrairement à la conception courante qui conçoit les vies comme « plates ». (shrink)