Utilizing a pre-experimental pre-test post-test design, this study investigated the effect of an authentic audio-visual American sitcom on the intercultural sensitivity and intercultural effectiveness of a sample of male and female upper-intermediate English students. To this aim, 34 Iranian EFL students were selected through convenient non-random sampling. In order to assure the participants' homogeneity in English proficiency, the selected students were given the Oxford Quick Placement Test prior to the intervention. Over a 10-week period, the participants were presented (...) with nearly 30-min-long episodes of Friends in a flipped context, two episodes every week uploaded to the accompanying website. An online 10-item quiz on the day of watching and a next-day meeting were held. During the online discussion meetings, the participants discussed cultural questions tailored to the aims of the study. The required data were collected through the administration of the intercultural sensitivity scale and intercultural effectiveness scale in pre- and post-test assessments. The analyses of the data indicated that both ICS and ICE improved significantly during the intervention. However, ICS indicated more improvement. In addition, a semi-structured interview was administered to the participants to examine their perception of the flipped classroom experience. Having analyzed the data using MAXQDA 2020, some significant themes emerged which are reported. On the whole, the participants portrayed positive attitudes toward using technology in flipped classes. The educational and practical implications are discussed. (shrink)

Online data collection with infants raises special opportunities and challenges for developmental research. One of the most prevalent methods in infancy research is eye-tracking, which has been widely applied in laboratory settings to assess cognitive development. Technological advances now allow conducting eye-tracking online with various populations, including infants. However, the accuracy and reliability of online infant eye-tracking remain to be comprehensively evaluated. No research to date has directly compared webcam-based and in-lab eye-tracking data from infants, similarly to data from adults. (...) The present study provides a direct comparison of in-lab and webcam-based eye-tracking data from infants who completed an identical looking time paradigm in two different settings. We assessed 4-6-month-old infants in an eye-tracking task that measured the detection of audio-visual asynchrony. Webcam-based and in-lab eye-tracking data were compared on eye-tracking and video data quality, infants’ viewing behavior, and experimental effects. Results revealed no differences between the in-lab and online setting in the frequency of technical issues and participant attrition rates. Video data quality was comparable between settings in terms of completeness and brightness, despite lower frame rate and resolution online. Eye-tracking data quality was higher in the laboratory than online, except in case of relative sample loss. Gaze data quantity recorded by eye-tracking was significantly lower than by video in both settings. In valid trials, eye-tracking and video data captured infants’ viewing behavior uniformly, irrespective of setting. Despite the common challenges of infant eye-tracking across experimental settings, our results point toward the necessity to further improve the precision of online eye-tracking with infants. Taken together, online eye-tracking is a promising tool to assess infants’ gaze behavior but requires careful data quality control. The demographic composition of both samples differed from the generic population on caregiver education: our samples comprised caregivers with higher-than-average education levels, challenging the notion that online studies will per se reach more diverse populations. (shrink)

This is a philosophical study by Lithuanian authors on issues related to how to teach philosophy, especially moral philosophy, through films, paintings, images, etc. The topics include multimodality as a synthesis; semiotics and language and image; cinema and philosophical education; postructuralism; film education; value education through spiritual cinema; Eastern Ethics for Western students through multimodal education; philosophy for children; sound and multimodality; Pedagogy of aesthetic to eco-pedagogy, etc.

People can discriminate the synchrony between audio-visual scenes. However, the sensitivity of audio-visual synchrony perception can be affected by many factors. Using a simultaneity judgment task, the present study investigated whether the synchrony perception of complex audio-visual stimuli was affected by audio-visual causality and stimulus reliability. In Experiment 1, the results showed that audio-visual causality could increase one's sensitivity to audio-visual onset asynchrony (AVOA) of both action stimuli and (...) speech stimuli. Moreover, participants were more tolerant of AVOA of speech stimuli than that of action stimuli in the high causality condition, whereas no significant difference between these two kinds of stimuli was found in the low causality condition. In Experiment 2, the speech stimuli were manipulated with either high or low stimulus reliability. The results revealed a significant interaction between audio-visual causality and stimulus reliability. Under the low causality condition, the percentage of “synchronous” responses of audio-visual intact stimuli was significantly higher than that of visual_intact/auditory_blurred stimuli and audio-visual blurred stimuli. In contrast, no significant difference among all levels of stimulus reliability was observed under the high causality condition. Our study supported the synergistic effect of top-down processing and bottom-up processing in audio-visual synchrony perception. (shrink)

In this paper we offer a theory of cross-modal objects. To begin, we discuss two kinds of linkages between vision and audition. The first is a duality. The the visual system detects and identifies surfaces ; the auditory system detects and identifies sources . Surfaces are illuminated by sources of light; sound is reflected off surfaces. However, the visual system discounts sources and the auditory system discounts surfaces. These and similar considerations lead to the Theory of Indispensable Attributes (...) that states the conditions for the formation of gestalts in the two modalities. The second linkage involves the formation of audiovisual objects, integrated cross-modal experiences. We describe research that reveals the role of cross-modal causality in the formation of such objects. These experiments use the canonical example of a causal link between vision and audition: a visible impact that causes a percussive sound. (shrink)

We examined whether facial expressions of performers influence the emotional connotations of sung materials, and whether attention is implicated in audio-visual integration of affective cues. In Experiment 1, participants judged the emotional valence of audio-visual presentations of sung intervals. Performances were edited such that auditory and visual information conveyed congruent or incongruent affective connotations. In the single-task condition, participants judged the emotional connotation of sung intervals. In the dual-task condition, participants judged the emotional connotation of (...) intervals while performing a secondary task. Judgements were influenced by melodic cues and facial expressions and the effects were undiminished by the secondary task. Experiment 2 involved identical conditions but participants were instructed to base judgements on auditory information alone. Again, facial expressions influenced judgements and the effect was undiminished by the secondary task. The results suggest that visual aspects of music performance are automatically and preattentively registered and integrated with auditory cues. (shrink)

Cerebral Visual Impairment is an umbrella term which includes abnormalities in visual acuity, or contrast sensitivity or colour; ocular motility; visual field and the conscious and unconscious filtering or processing of visual input. Children with CVI have specific needs and problems relating to their development from infancy to adulthood which can impact on their wellbeing. Recent research indicates the complexities of living with CVI but there remains limited information of the full impact of CVI on families’ (...) everyday lives. The qualitative interviews reported here explored families’ experiences to discover the impact of CVI on all aspects of everyday life. Parents and children were invited to participate in semi-structured interviews, either face to face, by phone or video call between January 2018 and February 2019. Topics covered everyday practicalities of living with CVI, focusing on challenges and what worked well at school and home. Interviews were audio-recorded and subject to thematic analysis to look for patterns across the data. Twenty families took part in interviews, with eight children/young people within those families contributing interviews of their own. Four themes were developed from the interviews: Assessment and understanding implications of CVI, Education, Family life, Psychological wellbeing and quality of life. The interviews provide valuable insights into the impact of living with CVI and highlight the need for more awareness of the condition among professionals in both health and education settings. (shrink)

The process of continuing education of paramedics is based on gaining educational credits during five-year educational periods. One of the forms of self-improvement are Internet-based educational programs. The lack of regulations concerning the organizational and technical aspects of e-learning made the authors attempt to analyze the phenomenon. The aim of the article is to present an initial analysis of the role of online educational programs in comparison with other forms of professional training of paramedics. One in three respondents has (...) recently used one of the methods of distance learning. Despite the low awareness of the effectiveness of e-learning, almost 70% of the interviewees expressed a positive opinion about it and over 50% were willing to use such forms in the future. The respondents preferred online trainings in the asynchronous version, containing audio-visual materials and providing the possibility to obtain a certificate issued to the participant after completing the course. The demand for topics in the area of professional training included mainly the fields of Drug Therapy, electrotherapy, electrocardiogram evaluation, medical segregation, and help in delivery in pre-hospital conditions. Modern forms of distance learning do not make it possible to acquire practical skills, yet their role in professional training of paramedics is still significant. According to the authors, online educational programs constitute an effective complement to traditional training through the performance of tasks in a mixed form, known as blended-learning. (shrink)

This paper has already been published in Sociological Research Online, 21/3, August 2016. All rights reserved. © SOCIOLOGICAL RESEARCH ONLINE. It is available from Kent Academic Repository. We thank Dawn Lyon for the permission to republish it here.: This article documents, shows and analyses the everyday rhythms of Billingsgate, London's wholesale fish market. It takes the form of a short film based an audiovisual montage of time-lapse photography and sound recordings, and a - Sociologie – Nouvel article.

In a century where humans and machines—powered by artificial intelligence or not—increasingly work together, it is of interest to understand human processing of multi-sensory stimuli in relation to attention and working memory. This paper explores whether and when supporting visual information with rhythmic auditory stimuli can optimize multi-sensory information processing. In turn, this can make the interaction between humans or between machines and humans more engaging, rewarding and activating. For this purpose a novel working memory paradigm was developed where (...) participants are presented with a series of five target digits randomly interchanged with five distractor digits. Their goal is to remember the target digits and recall them orally. Depending on the condition support is provided by audio and/or rhythm. It is expected that the sound will lead to a better performance. It is also expected that this effect of sound is different in case of rhythmic and non-rhythmic sound. Last but not least, some variability is expected across participants. To make correct conclusions, the data of the experiment was statistically analyzed in a classic way, but also predictive models were developed in order to predict outcomes based on a range of input variables related to the experiment and the participant. The effect of auditory support could be confirmed, but no difference was observed between rhythmic and non-rhythmic sounds. Overall performance was indeed affected by individual differences, such as visual dominance or perceived task difficulty. Surprisingly a music education did not significantly affect the performance and even tended toward a negative effect. To better understand the underlying processes of attention, also brain activation data, e.g., by means of electroencephalography, should be recorded. This approach can be subject to a future work. (shrink)

Experimental research has shown that pairs of stimuli which are congruent and assumed to 'go together' are recalled more effectively than an item presented in isolation. Will this multisensory memory benefit occur when stimuli are richer and longer, in an ecological setting? In the present study, we focused on an everyday situation of audio-visual learning and manipulated the relationship between audio guide tracks and viewed portraits in the galleries of the Tate Britain. By varying the gender and (...) narrative style of the voice-over, we examined how the perceived congruency and assumed unity of the audio guide track with painted portraits affected subsequent recall. We show that tracks perceived as best matching the viewed portraits led to greater recall of both sensory and linguistic content. We provide the first evidence that manipulating crossmodal congruence and unity assumptions can effectively impact memory in a multisensory ecological setting, even in the absence of precise temporal alignment between sensory cues. (shrink)

It is commonly believed that human perceptual experiences can be, and usually are, multimodal. What is more, a stronger thesis is often proposed that some perceptual multimodal characters cannot be described simply as a conjunction of unimodal phenomenal elements. If it is the case, then a question arises: what is the additional mode of combination that is required to adequately describe the phenomenal structure of multimodal experiences? The paper investigates what types of audio–visual experiences have phenomenal character that (...) cannot be analysed as a mere conjunction of visual and auditory elements; and how can we properly characterise the required, additional mode of perceptual combination. Three main modes of combination are considered: instantiation, parthood, and grouping. It is argued that some phenomena involving intermodal relations, like spatial and temporal ventriloquism, can be analysed in terms of audio–visual, perceptual grouping. On the other hand, cases of intermodal binding need a different treatment. Experiences involving audio–visual binding should be analysed as experiences presenting objects or events which instantiate, or which have a proper part instantiating, both visually and auditorily determined properties. (shrink)

This article investigates the hypothesis of creating non-linear audio-visual narratives, through an unanticipated use of traditional print-based games, enriched with videos, via augmented reality (AR) possibilities. A ludic system has been created and presented. Based on a traditional card game, a non-linear cinematic narrative occurs. We attempt to examine the following questions: in which way can we bring together different forms of visual communication, such as graphic design and video? Can the above forms create a complex narrative (...) whole and what kind of rules will be needed for this? How can we enrich traditional forms of gaming with the potentials of AR? Gaming itself demands a set of rules. Can these rules play the role of algorithms in the combined universe that we have designed and created? In which way can the designer on the one hand and the user on the other influence the overall output of the system? What will the user experience be like? The printed card system chosen for this is Tarot and more precisely the Great Arcana, which makes use of the 22 fundamental Tarot figures. (shrink)

We investigate whether smile mimicry and emotional contagion are evident in non-text-based computer-mediated communication (CMC). Via an ostensibly real-time audio-visual CMC platform, participants interacted with a confederate who either smiled radiantly or displayed a neutral expression throughout the interaction. Automatic analyses of expressions displayed by participants indicated that smile mimicry was at play: A higher level of activation of the facial muscle that characterises genuine smiles was observed among participants who interacted with the smiling confederate than among participants (...) who interacted with the unexpressive confederate. However, there was no difference in the self-reported level of joviality between participants in the two conditions. Our findings demonstrate that people mimic smiles in audio-visual CMC, but that even though the diffusion of emotions has been documented in text-based CMC in previous studies, we find no convincing support for the phenomenon of emotional contagion in non-text-based CMC. (shrink)

Previous research with speech and non-speech stimuli suggested that in audiovisual perception, visual information starting prior to the onset of corresponding sound can provide visual cues, and form a prediction about the upcoming auditory sound. This prediction leads to audiovisual interaction. Auditory and visual perception interact and induce suppression and speeding up of the early auditory event-related potentials such as N1 and P2. To investigate AV interaction, previous research examined N1 and P2 amplitudes and latencies in response (...) to audio only, video only, audiovisual, and control stimuli, and compared AV with auditory perception based on four AV interaction models. The current study addresses how different models of AV interaction express N1 and P2 suppression in music perception. Furthermore, the current study took one step further and examined whether previous musical experience, which can potentially lead to higher N1 and P2 amplitudes in auditory perception, influenced AV interaction in different models. Musicians and non-musicians were presented the recordings of a keyboard /C4/ key being played, as well as CO stimuli. Results showed that AV interaction models differ in their expression of N1 and P2 amplitude and latency suppression. The calculation of model and has consequences for the resulting N1 and P2 difference waves. Furthermore, while musicians, compared to non-musicians, showed higher N1 amplitude in auditory perception, suppression of amplitudes and latencies for N1 and P2 was similar for the two groups across the AV models. Collectively, these results suggest that when visual cues from finger and hand movements predict the upcoming sound in AV music perception, suppression of early ERPs is similar for musicians and non-musicians. Notably, the calculation differences across models do not lead to the same pattern of results for N1 and P2, demonstrating that the four models are not interchangeable and are not directly comparable. (shrink)

The purpose of this study is to discuss the effects of audiovisual input on second language acquisition and the factors that influence the difficulty of audiovisual learning through a systematic literature review. Prior to this systematic review, in this paper, we searched papers on related topics for the past 10 years from 2012 to 2022, and found 46 journal papers that met the research criteria. They can basically represent the scholarly work in this field. The 46 studies were published in (...) journals indexed in Google Scholar, Eric, Scopus, and Wiley Library. Databases were selected according to a set of inclusion and exclusion criteria. The following conclusions are drawn from the literature review: Audiovisual input can provide more authentic language input and more adequate and richer multimodal cultural and situational contexts, which can better promote learners' understanding of the content and stimulate learners' interest in participating in listening comprehension tasks. The influencing factors of multimodal input on listening difficulty include subtitles, video type, and the relationship between the audio and visual input. (shrink)

This paper engages with Italo Calvino’s lecture on Visibility, included in his last—and testamentary—volume Six Memos, by understanding it in an educational and pedagogical key. While the question of pedagogy is expressly addressed by Calvino himself in his lecture, the interpretation here provided is not merely an application of his tenets but an elaboration on and an autonomous development of them. In particular, in the spotlight there is the intimate bond image-cum-writing which seems to preside over Calvino’s insights and is (...) here suggested as key to tackling the challenges of the contemporary mediascape and the “tautological vision” dominating therein. While a part of the educational discourse invokes “homeostatic” pedagogical solutions, namely the (ultimately confrontational) deployment of writing to compensate for the iconic ruling regime, this paper explores the possibility of a specific kind of visual education (instantiated by comics and Otto Neurath’s Isotype), which combines the role of the image with some features traditionally attributed to the pedagogy of writing (e.g. the cultivation of abilities of abstraction, of reflection etc.). A pedagogy of figuration is, accordingly, proposed as an interruption of the tautological vision of the new media and as conducive to educating readers of the unwritten world and of the world of digital images, Mr. Palomar—the hero of Calvino’s last novel—possibly being the archetype of this kind of (new?) readership. By referring to two influential notions in the contemporary debate in educational philosophy and theory, this pedagogy is finally interpreted in term of (visual) thing-centredness rather than (visual) subjectification. (shrink)

Lip reading is the ability to partially understand speech by looking at the speaker's lips. It improves the intelligibility of speech in noise when audio-visual perception is compared with audio-only perception. A recent set of experiments showed that seeing the speaker's lips also enhances sensitivity to acoustic information, decreasing the auditory detection threshold of speech embedded in noise [J. Acoust. Soc. Am. 109 (2001) 2272; J. Acoust. Soc. Am. 108 (2000) 1197]. However, detection is different from comprehension, (...) and it remains to be seen whether improved sensitivity also results in an intelligibility gain in audio-visual speech perception. In this work, we use an original paradigm to show that seeing the speaker's lips enables the listener to hear better and hence to understand better. The audio-visual stimuli used here could not be differentiated by lip reading per se since they contained exactly the same lip gesture matched with different compatible speech sounds. Nevertheless, the noise-masked stimuli were more intelligible in the audio-visual condition than in the audio-only condition due to the contribution of visual information to the extraction of acoustic cues. Replacing the lip gesture by a non-speech visual input with exactly the same time course, providing the same temporal cues for extraction, removed the intelligibility benefit. This early contribution to audio-visual speech identification is discussed in relationships with recent neurophysiological data on audio-visual perception. (shrink)

Companies increasingly seek to use gay protagonists in audio-visual commercials to attract a new affluent target group. There is also growing demand for the diversity present in society to be reflected in media formats such as advertising. Studies have shown, however, that heterosexual consumers, who may be part of the company's loyal consumer base, tend to react negatively to gay-themed advertising campaigns. Searching for an instrument to mitigate this unwanted effect, the present study investigated whether carefully selected background (...) music can shape the perceived gender of gay male advertising protagonists. In a 2 × 2 between-subjects online experiment, 218 heterosexual participants watched a commercial promoting engagement rings that featured gay male protagonists, scored with feminine- or masculine-connoted background music. As expected, women generally reacted more positively than men to the advertising. Men exposed to the masculine-connoted background music rated the promoted brand more positively, and masculine music also enhanced these men's acceptance of gay men in general more than was the case for feminine background music. Carefully selected background music affecting the perceived gender of gay male advertising protagonists may prevent negative reactions from heterosexual audiences and, therefore, motivate companies to use gay protagonists in television commercials on a more regular basis. (shrink)

This article provides an ethical analysis of the moral conflict over access to premium audio-visual online content, which is currently regulated by geo-blocking in the domain of digital film distri...

In 2015 the first mass migration and the first geopolitical and transnational crisis of the digital age began. This brought along the awakening of a new phase in slavery trade and trafficking of human beings (THB) at the European borders. In this article I introduce an ongoing field study I began in January 2016, in collaboration with experts from Spanish and European institutions, that so far has taken me to hotspots and borders, and to the accumulation of a vast multimedia (...) archive. This led to the creation of zonaautonomamilitarizada.eu, an immersive/360°, interactive and reactive audio-visual performative system conceptually rooted in military technologies, second-order cybernetics and mass media developments. (shrink)

During the COVID-19 pandemic, questions have been raised about the impact of face masks on communication in classroom settings. However, it is unclear to what extent visual obstruction of the speaker’s mouth or changes to the acoustic signal lead to speech processing difficulties, and whether these effects can be mitigated by semantic predictability, i.e., the availability of contextual information. The present study investigated the acoustic and visual effects of face masks on speech intelligibility and processing speed under varying (...) semantic predictability. Twenty-six children and twenty-six adults performed an internet-based cued shadowing task, in which they had to repeat aloud the last word of sentences presented in audio-visual format. The results showed that children and adults made more mistakes and responded more slowly when listening to face mask speech compared to speech produced without a face mask. Adults were only significantly affected by face mask speech when both the acoustic and the visual signal were degraded. While acoustic mask effects were similar for children, removal of visual speech cues through the face mask affected children to a lesser degree. However, high semantic predictability reduced audio-visual mask effects, leading to full compensation of the acoustically degraded mask speech in the adult group. Even though children did not fully compensate for face mask speech with high semantic predictability, overall, they still profited from semantic cues in all conditions. Therefore, in classroom settings, strategies that increase contextual information such as building on students’ prior knowledge, using keywords, and providing visual aids, are likely to help overcome any adverse face mask effects. (shrink)

During the interwar period, the encyclopaedia became a popular educative instrument for demonstrating knowledge. Within the field of cultural internationalism, the pioneer of documentation Paul Otlet redefined the encyclopaedia as a documentary product or as we would say today a "multi-media" product. This article discusses the exchange of ideas between Otlet, Patrick Geddes and Otto Neurath and shows how the graphic and scenographic demonstration of encyclopaedic knowledge at the beginning of the twentieth century applied the values of scientiic universalism to (...) programs of international education and cultural reform. (shrink)

This paper revisits some early applications of audio-visual imaging technologies used in physiology in a dialogue with reflections on Henri Bergson’s philosophy. It focuses on the aspects of time and memory in relation to spatial representations of movement measurements and critically discusses them from the perspective of the observing participant and the public exhibitions of scientific films. Departing from an audio-visual example, this paper is informed by a thick description of the philosophical implications and contemporary discourses (...) surrounding the scientific inventions, technologies and theories of moving image technologies, representations of time, and the measurement of bodies in motion such as those by the French physiologist Étienne-Jules Marey. The application of Bergson’s thinking will show how the scientific graphing and representations of the body through technologies inevitably stand in tension with the limits and potentials of the ‘human apparatus’ and its abilities, filters and internalised processes of perception, memory and consciousness. It proposes an ontological approach to scientific imaging technologies and a critical engagement with the ‘realism’ debate in the discourse of audio-visual media. (shrink)

ABSTRACTWe often use our own emotions to understand other people’s emotions. However, emotional egocentric biases, namely the tendency to use one’s own emotional state when relating to others...