It is often assumed that the reaction time of a saccade toward visual and/or auditory stimuli reflects the sensitivities of our oculomotor-orienting system to stimulus saliency. Endogenous factors, as well as stimulus-related factors, would also affect the saccadic reaction time. However, it was not clear how these factors interact and to what extent visual and auditory-targeting saccades are accounted for by common mechanisms. The present study examined the effect of, and the interaction between, stimulus (...) saliency and audiovisual spatial congruency on the SRT for visual- and for auditory-target conditions. We also analyzed pre-target pupil size to examine the relationship between saccade preparation and pupil size. Pupil size is considered to reflect arousal states coupling with locus-coeruleus activity during a cognitive task. The main findings were that the pattern of the examined effects on the SRT varied between visual- and auditory-auditory target conditions, the effect of stimulus saliency was significant for the visual-target condition, but not significant for the auditory-target condition, Pupil velocity, not absolute pupil size, was sensitive to task set, and there was a significant correlation between the pre-saccade absolute pupil size and the SRTs for the visual-target condition but not for the auditory-target condition. The discrepancy between target modalities for the effect of pupil velocity and between the absolute pupil size and pupil velocity for the correlation with SRT may imply that the pupil effect for the visual-target condition was caused by a modality-specific link between pupil size modulation and the SC rather than by the LC-NE system. These results support the idea that different threshold mechanisms in the SC may be involved in the initiation of saccades toward visual and auditory targets. (shrink)

This study examined self-recognition processing in both the auditory and visual modalities by determining how comparable hearing a recording of one’s own voice was to seeing photograph of one’s own face. We also investigated whether the simultaneous presentation of auditory and visual self-stimuli would either facilitate or inhibit self-identification. Ninety-one participants completed reaction-time tasks of self-recognition when presented with their own faces, own voices, and combinations of the two. Reaction time and errors made when (...) responding with both the right and left hand were recorded to determine if there were lateralization effects on these tasks. Our findings showed that visual self-recognition for facial photographs appears to be superior to auditory self-recognition for voice recordings. Furthermore, a combined presentation of one’s own face and voice appeared to inhibit rather than facilitate self-recognition and there was a left-hand advantage for reaction time on the combined-presentation tasks. (shrink)

To further develop three-dimensional (3D) applications, it is important to elucidate the negative effects of 3D applications on the human body and mind. Thus, this study investigated differences in the effects of visual fatigue on cognition and brain activity using visual and auditory tasks induced by watching a 1-h movie in two dimensions (2D) and 3D. Eighteen young men participated in this study. Two conditions were randomly performed for each participant on different days, namely, watching (...) the 1-h movie on television in 2D (control condition) and 3D (3D condition). Before and after watching the 1-h movie on television, critical flicker fusion frequency (CFF: an index of visual fatigue), and response accuracy and reaction time for the cognitive tasks were determined. Brain activity during the cognitive tasks was evaluated using a multi-channel near-infrared spectroscopy system. In contrast to the control condition, the decreased CFF, and the lengthened reaction time and the decreased activity around the right primary somatosensory cortex during Go/NoGo blocks in the visual task at post-viewing in the 3D condition were significant, with significant repeated measures correlations among them. Meanwhile, in the auditory task, the changes in cognitive performance and brain activity during the Go/NoGo blocks were not significant in the 3D condition. These results suggest that the failure or delay in the transmission of visual information to the primary somatosensory cortex due to visual fatigue induced by watching a 3D movie reduced the brain activity around the primary somatosensory cortex, resulting in poor cognitive performance for the visual task. This suggests that performing tasks that require visual information, such as running in the dark or driving a car, immediately after using a 3D application, may create unexpected risks in our lives. Thus, the findings of this study will help outlining precautions for the use of 3D applications. (shrink)

There are indications that simple reaction time might have slowed in Western countries, based on both cohort- and multi-study comparisons. A possible limitation of the latter method in particular is measurement error stemming from methods variance, which results from the fact that instruments and experimental conditions change over time and between studies. We therefore set out to measure the simple auditory reaction time (SRT) of 7,081 individuals (2,997 males and 4,084 females) born in Sweden 1959-1985 (subjects were (...) aged between 27 and 54 years at time of measurement). Depending on cut-offs and adjustment for ageing related slowing on SRT, the data suggest that SRT has increased between 3 and 16 ms in the 27 birth years covered in the present sample. The slowing is unlikely to be explained by attrition, as evaluated by comparing the general intelligence × birth-year interactions and standard deviations for both male participants and dropouts, utilizing military conscript cognitive ability data. The present result is consistent with previous studies employing alternative methods, and may result from several synergistic factors, such as possible recent micro-evolutionary trends favouring lower g in Sweden and the effects of industrially produced neurotoxic substances on peripheral nerve conduction velocity. (shrink)

People are ingesting various information from different sense organs all the time to complete different cognitive tasks. The brain integrates and regulates this information. The two significant sensory channels for receiving external information are sight and hearing that have received extensive attention. This paper mainly studies the effect of music and visual-auditory stimulation on electroencephalogram of happy emotion recognition based on a complex system. In the experiment, the presentation was used to prepare the experimental stimulation program, and the (...) cognitive neuroscience experimental paradigm of EEG evoked by happy emotion pictures was established. Using 93 videos as natural stimuli, fMRI data were collected. Finally, the collected EEG signals were removed with the eye artifact and baseline drift, and the t-test was used to analyze the significant differences of different lead EEG data. Experimental data shows that, by adjusting the parameters of the convolutional neural network, the highest accuracy of the two-classification algorithm can reach 98.8%, and the average accuracy can reach 83.45%. The results show that the brain source under the combined visual and auditory stimulus is not a simple superposition of the brain source of the single visual and auditory stimulation, but a new interactive source is generated. (shrink)

IntroductionAlthough the motor signs of Parkinson’s disease are well defined, nonmotor symptoms, including higher-level language deficits, have also been shown to be frequent in patients with PD. In the present study, we used a lexical decision task to find out whether access to the mental lexicon is impaired in patients with PD, and whether task performance is affected by bradykinesia.Materials and MethodsParticipants were 34 nondemented patients with PD, either without medication or under optimum medication. A total of 19 age-matched control (...) volunteers were also recruited. We recorded reaction times to the LDT and a simple RT task. In each task, stimuli were either visual or auditory. Statistical analyses consisted of repeated-measures analyses of variance and Tukey’s HSD post hoc tests.ResultsIn the LDT, participants with PD both off and on medication exhibited intact access to the mental lexicon in both modalities. In the visual modality, patients off medication were just as fast as controls when identifying real words, but slower when identifying pseudowords. In the visual modality of the control task, RTs for pseudowords were significantly longer for PD patients off medication than for controls, revealing an unexpected but significant lexicality effect in patients that was not observed in the auditory modality. Performances of patients on medication did not differ from those of age-matched controls.DiscussionMotor execution was not slowed in patients with PD either off or on medication, in comparison with controls. Regarding lexical access, patients off medication seemed to have difficulty inhibiting a cognitive-linguistic process when it was not required, and exhibit a specific pseudoword processing deficit in the LDT, which may have been related to impaired lateral word inhibition within the mental lexicon. These deficits seemed to be compensated by medication. (shrink)

In three macaque monkeys with unilateral removal of primary visual cortex and in one unoperated monkey, we measured reaction times to a visual target that was presented at a lateral eccentricity of 20o in the normal, left, visual hemifield. When an additional stimulus was presented at the corresponding position in the right hemifield (hemianopic in three of the monkeys), it significantly slowed the reaction time to the left target if it preceded it by delays from (...) 100-500 msec. The most effective delay depended on the particular experimental paradigm and perhaps on the experience of the monkey with the task. The results show that reaction times to seen targets in the normal hemifield of monkeys are influenced by the presentation of ''unseen'' targets in the anopic hemifield, as in some patients with cortically blind visual field defects. (shrink)

Auditory-guided vocal learning is a mechanism that operates both in humans and other animal species making us capable to imitate arbitrary sounds. Both auditory memories and auditory feedback interact to guide vocal learning. This may explain why it is easier for humans to imitate the pitch of a human voice than the pitch of a synthesized sound. In this study, we compared the effects of two different feedback modalities in learning pitch-matching abilities using a synthesized pure (...) tone in 47 participants with no prior music experience. Participants were divided into three groups: a feedback group receiving real-time visual feedback of their pitch as well as knowledge of results; an equal-timbre group receiving additional auditory feedback of the target note with a similar timbre to the instrument being used ; and a control group practicing without any feedback or knowledge of results. An additional fourth group of violin experts performed the same task for comparative purposes. All groups were posteriorly evaluated in a transfer phase. Both experimental groups improved their intonation abilities with the synthesized sound after receiving feedback. Participants from the equal-timber group seemed as capable as the feedback group of producing the required pitch with the voice after listening to the human voice, but not with the violin. In addition, only participants receiving real-time visual feedback learned and retained in the transfer phase the mapping between the synthesized pitch and its correspondence with the produced vocal or violin pitch. It is suggested that both the effect of an objective external reward, together with the experience of exploring the pitch space with their instrument in an explicit manner, helped participants to understand how to control their pitch production, strengthening their schemas, and favoring retention. (shrink)

Learning to play a musical instrument involves mapping visual + auditory cues to motor movements and anticipating transitions. Inspired by the serial reaction time task and artificial grammar learning, we investigated explicit and implicit knowledge of statistical learning in a sensorimotor task. Using a between-subjects design with four groups, one group of participants were provided with visual cues and followed along by tapping the corresponding fingertip to their thumb, while using a computer glove. Another group additionally (...) received accompanying auditory tones; the final two groups received sensory cues but did not provide a motor response—all together following a 2 × 2 design. Implicit knowledge was measured by response time, whereas explicit knowledge was assessed using probe tests. Findings indicate that explicit knowledge was best with only the single modality, but implicit knowledge was best when all three modalities were involved. (shrink)

Our research aimed to investigate the effectiveness of auditory, visual, and audiovisual warning signals for capturing the attention of the pilot, and how stimulus onset asynchronies in audiovisual stimuli affect pilots perceiving the bimodal warning signals under different perceptual load conditions. In experiment 1 of the low perceptual load condition, participants discriminated the location of visual targets preceded by five different types of warning signals. In experiment 2 of high perceptual load, participants completed the location task identical (...) to a low load condition and a digit detection task in a rapid serial visual presentation stream. The main effect of warning signals in two experiments showed that visual and auditory cues presented simultaneously could effectively and efficiently arouse the attention of the pilots in high and low load conditions. Specifically, auditory, AV, and visual preceding auditory stimulus by 100 ms increased the spatial orientation to a valid position in low load conditions. With the increase in visual perceptual load, auditory preceding the visual stimulus by 100 ms and A warning signals had stronger spatial orientation. The results are expected to theoretically support the optimization design of the cockpit display interface, contributing to immediate flight crew awareness. (shrink)

This investigation examined age-related differences in auditory-visual integration as reflected on perceptual judgments of temporally misaligned AV English sentences spoken by native English and native Spanish talkers. In the detection task, it was expected that slowed auditory temporal processing of older participants, relative to younger participants, would be manifest as a shift in the range over which participants would judge asynchronous stimuli as synchronous. The older participants were also expected to exhibit greater declines in speech recognition for (...) asynchronous AV stimuli than younger participants. Talker accent was hypothesized to influence listener performance, with older listeners exhibiting a greater narrowing of the AV simultaneity window and much poorer recognition of asynchronous AV foreign-accented speech compared to younger listeners. Participant groups included younger and older participants with normal hearing and older participants with hearing loss. Stimuli were video recordings of sentences produced by native English and native Spanish talkers. The video recordings were altered in 50 ms steps by delaying either the audio or video onset. Participants performed a detection task in which they judged whether the sentences were synchronous or asynchronous, and performed a recognition task for multiple synchronous and asynchronous conditions. Both the detection and recognition tasks were conducted at the individualized signal-to-noise ratio corresponding to approximately 70% correct speech recognition performance for synchronous AV sentences. Older listeners with and without hearing loss generally showed wider AV simultaneity windows than younger listeners, possibly reflecting slowed auditory temporal processing in auditory lead conditions and reduced sensitivity to asynchrony in auditory lag conditions. However, older and younger listeners were affected similarly by misalignment of auditory and visual signal onsets on the speech recognition task. This suggests that older listeners are negatively impacted by temporal misalignments for speech recognition, even when they do not notice that the stimuli are asynchronous. Overall, the findings show that when listener performance is equated for simultaneous AV speech signals, age effects are apparent in detection judgments but not in recognition of asynchronous speech. (shrink)

Processing the motivational relevance of a visual scene and reacting accordingly is crucial for survival. Previous work suggests the emotional content of naturalistic scenes affects response speed, such that unpleasant content slows responses whereas pleasant content accelerates responses. It is unclear whether these effects reflect motor-cognitive processes, such as attentional orienting, or vary with the function/outcome of the motor response itself. Four experiments manipulated participants’ ability to terminate the picture and, thereby, the response’s function and motivational value. Attentive (...) orienting was manipulated via picture repetition, which diminishes orienting. A total of N = 81 participants completed versions of a go/no-go task, discriminating between distorted versus intact pictures drawn from six content categories varying in positive, negative, or neutral valence. While all participants responded faster with repetition, only participants without offset control exhibited slower responses to unpleasant and accelerated responses to pleasant content. Emotional engagement, measured by the late positive potential, was not modulated by attentional orienting, suggesting that the interaction between repetition and offset control is not due to altered emotional engagement. Together, results suggest that response time changes as a function of emotional content and sensitivity to attention orienting depends on the motivational function of the motor response. (shrink)

Many studies have argued for distinct but complementary contributions from each hemisphere in the control of movements to visual targets. Investigators have attempted to extend observations from patients with unilateral left- and right-hemisphere damage, to those using neurologically-intact participants, by assuming that each hand has privileged access to the contralateral hemisphere. Previous attempts to illustrate right hemispheric contributions to the control of aiming have focussed on increasing the spatial demands of an aiming task, to attenuate the typical right hand (...) advantages, to try to enhance a left hand reaction time advantage in right-handed participants. These early attempts have not been successful. The present study circumnavigates some of the theoretical and methodological difficulties of some of the earlier experiments, by using three different tasks linked directly to specialized functions of the right hemisphere: bisecting, the gap effect, and visuospatial localization. None of these tasks were effective in reducing the magnitude of left hand reaction time advantages in right handers. Results are discussed in terms of alternatives to right hemispheric functional explanations of the effect, the one-dimensional nature of our target arrays, power and precision given the size of the left hand RT effect, and the utility of examining the proportions of participants who show these effects, rather than exclusive reliance on measures of central tendency and their associated null hypothesis significance tests. (shrink)

Although substantial evidence demonstrates that directing attention to specific items is important for improving the performance of visual working memory, it is still not clear whether the attended items were better protected. The present study, thus, adopted a pre-cueing paradigm to examine the effect of attention direction and perceptual distractor on VWM. The results showed that a valid visual cue improved the individuals’ VWM performances and reduced their reaction time compared to the invalid and neutral cues. However, (...) the VWM performances in the valid and neutral cue conditions were more disrupted by a post-stimuli distractor compared to the invalid cue condition. The findings suggest that although directing attention can improve the VWM performance, it is not efficient in protecting information from being distracted. (shrink)

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of (...) early cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing. (shrink)

PurposeThis study investigated the impact of age and myopia on visual form perception among Chinese school-age children.MethodsThis cross-sectional study included 1,074 students with a mean age of 12.1 ± 4.7 years. The mean spherical equivalence refraction of the participants was −1.45 ± 2.07 D. All participants underwent distance visual acuity, refraction measurement and local and global visual form perception test including orientation, parallelism, collinearity, holes and color discrimination tasks.ResultsThe reaction times of emmetropes were slower than those (...) of myopic and high myopic groups on both local and global discrimination tasks. A reduction in reaction times was found with increasing age on both local and global discrimination tasks. Age was significantly associated with both local and global visual perception performance after adjusting for gender, visual acuity and SER.ConclusionsThis study revealed that both local and global visual perception improve with age among Chinese children and that myopes seem to have better visual perception than emmetropes. (shrink)

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of (...) early cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing. (shrink)

The redundant-signals paradigm (RSP) is designed to investigate response behavior in perceptual tasks in which response-relevant targets are defined by either one or two features, or modalities. The common finding is that responses are speeded for redundantly compared to singly defined targets. This redundant-signals effect (RSE) can be accounted for by race models if the response times do not violate the race model inequality (RMI). When there are violations of the RMI, race models are effectively excluded as a viable account (...) of the RSE. The common alternative is provided by co-activation accounts, which assume that redundant target signals are integrated at some processing stage. However, ‘co-activation’ has mostly been only indirectly inferred and the accounts have only rarely been explicitly modeled; if they were modeled, the RSE has typically been assumed to have a decisional locus. Yet, there are also indications in the literature that the RSE might originate, at least in part, at a non-decisional or motor stage. In the present study, using a distribution analysis of sequential-sampling models (ex-Wald and Ratcliff Diffusion model), the locus of the RSE was investigated for two bimodal (audio-visual) detection tasks that strongly violated the RMI, indicative of substantial co-activation. Three model variants assuming different loci of the RSE were fitted to the quantile reaction time proportions: a decision, a non-decision, and a combined variant both to vincentized group as well as individual data. The results suggest that for the two bimodal detection tasks, co-activation has a shared decisional and non-decisional locus. These findings point to the possibility that the mechanisms underlying the RSE depend on the specifics (task, stimulus, conditions, etc.) of the experimental paradigm. (shrink)

Two essential components make up the semantic information that was used to pinpoint the video events. They consist of: (a) A spatial component of a video frame, such as the scenery, visible people, and visible objects. (b) A temporal component that is represented by a series of video frames across time, such as the actions of a character or the movements of an object. The video's audio, video, and text components are examined in order to provide the higher-level semantic data. (...) The semantic structure of the movie is discovered using a variety of cues from many modalities, such as auditory and visual elements, bridging the gap between high level semantic notions and low level qualities. A video contains three distinct modalities. The three are as follows: (1) Visual modality, which refers to the scene seen in the video; (2) Auditory modality, which refers to spoken words, background music, and other noises; and (3) Textual modality, which refers to written materials that describe the subject matter of the movie. There are numerous video databases and a sizable amount of video data in general. Even while there are tools for maintaining and exploring these collections, many decision-making applications increasingly require tools to extract the critical information hidden in the video data. (shrink)

Asymmetric distribution of function between the cerebral hemispheres has been widely investigated in the auditory modality. The current approach borrows heavily from visual local–global research in an attempt to determine whether, as in vision, local–global auditory processing is lateralised. In vision, lateralised local–global processing likely relies on spatial frequency information. Drawing analogies between visual spatial frequency and auditory dimensions, two sets of auditory stimuli were developed. In the high–low stimulus set we manipulate frequency information, (...) and in the fast–slow stimulus set we manipulate temporal information. The fast–slow stimuli additionally mimic visual hierarchical stimulus structure, in which the arrangement of local patterns determines the global pattern. Unlike previous auditory stimuli, the current stimulus sets contain the experimental flexibility of visual local–global hierarchical stimuli allowing independent manipulation of structural levels. Previous findings of frequency and temporal range priming were replicated. Additionally, by presenting stimuli monaurally, we found that priming of frequency ranges (but not temporal ranges) was found to vary by ear, supporting the contention that the hemispheres asymmetrically retain traces of prior frequency processing. These results contribute to the extensive literature revealing cerebral asymmetries for the processing of frequency information, and extend those results to the realm of priming. (shrink)

The coronavirus pandemic has resulted in the recommended/required use of face masks in public. The use of a face mask compromises communication, especially in the presence of competing noise. It is crucial to measure the potential effects of wearing face masks on speech intelligibility in noisy environments where excessive background noise can create communication challenges. The effects of wearing transparent face masks and using clear speech to facilitate better verbal communication were evaluated in this study. We evaluated listener (...) word identification scores in the following four conditions: (1) type of mask condition (i.e., no mask, transparent mask, and disposable face mask), (2) presentation mode (i.e., auditory only and audiovisual), (3) speaking style (i.e., conversational speech and clear speech), and (4) with two types of background noise (i.e., speech shaped noise and four-talker babble at −5 signal-to-noise ratio). Results indicate that in the presence of noise, listeners performed less well when the speaker wore a disposable face mask or a transparent mask compared to wearing no mask. Listeners correctly identified more words in the audiovisual presentation when listening to clear speech. Results indicate the combination of face masks and the presence of background noise negatively impact speech intelligibility for listeners. Transparent masks facilitate the ability to understand target sentences by providing visual information. Use of clear speech was shown to alleviate challenging communication situations including compensating for a lack of visual cues and reduced acoustic signals. (shrink)