The contemporary design practice known as data sonification allows us to experience information in data by listening. In doing so, we understand the source of the data in ways that support, and in some cases surpass, our ability to do so visually. -/- In order to assist us in negotiating our environments, our senses have evolved differently. Our hearing affords us unparalleled temporal and locational precision. Biological survival has determined that the ears lead the eyes. For all moving creatures, (...) in situations where sight is obscured, spatial auditory clarity plays a vital survival role in determining both from where the predator is approaching or to where the prey has escaped. So, when designing methods that enable listeners to extract information from data, both with and without visual support, different approaches are necessary. -/- A scholarly yet approachable work by one of the recognized leaders in the field of auditory design, this book will -/- - Lead you through some salient historical examples of how non-speech sounds have been used to inform and control people since ancient times. -/- - Comprehensively summarize the contemporary practice of Data Sonification. -/- - Provide a detailed overview of what information is and how our auditory perceptions can be used to enhance our knowledge of the source of data. -/- - Show the importance of the dynamic relationships between hearing, cognitive load, comprehension, embodied knowledge and perceptual truth. -/- - Discuss the role of aesthetics in the dynamic interplay between listenability and clarity. -/- - Provide a mature software framework that supports the practice of data sonification design, together with a detailed discussion of some of the design principles used in various examples. -/- David Worrall is an internationally recognized composer, sound artist and interdisciplinary researcher in the field of auditory design. He is Professor of Audio Arts and Acoustics at Columbia College Chicago and a former elected president of the International Community for Auditory Display (ICAD), the leading organization in the field since its inception over 25 years ago. -/- Code and audio examples for this book are available at -/- https:// github. com/david-worrall/springer/. (shrink)

Sound can be listened to in various ways and with different intentions. Multiple factors influence how and what we perceive when listening to sound. Sonification, the acoustic representation of data, is in essence just sound. It functions as sonification only if we make sure to listen attentively in order to access the abstract information it contains. This is difficult to accomplish since sound always calls the listener’s attention to concrete—whether natural or musical—points of references. Important aspects determining how (...) we listen to sonification are discussed in this paper: elicited sounds, repeated sounds, conceptual sounds, technologically mediated sounds, melodic sounds, familiar sounds, multimodal sounds and vocal sounds. We discuss how these aspects help the listener engage with the sound, but also how they can become points of reference in and of themselves. The various sonic qualities employed in sonification can potentially open but also risk closing doors to the accessibility and perceptibility of the sonified data. (shrink)

The definition of sonification has been reframed in recent years but remains somewhat in flux; the basic concepts and procedural flows have remained relatively unchanged. Recent definitions have focused on the objective the important uses of sonification in terms of scientific method. The full realization of the potential of the field must also include the craft and art of music composition. The author proposes examining techniques of sonification in a two-order framework: direct and procedural. The impact of (...) new technologies and historical roots of that work argues that framing this broad topic should be in terms inclusive of scientific method and craftsmanship and art. The expressive use of sonic time-based data flows needs to be refined and expanded. The unexamined territory of how a broad-based population of listeners on a subjective, as well as objective level needs, have to be included in this new field. (shrink)

The public release of datasets on the internet by government agencies, environmental scientists, political groups and many other organizations has fostered a social practice of data visualization. The audiences have expectations of production values commensurate with their daily experience of professional visual media. At the same time, access to this data has allowed visual designers and artists to apply their skills to what was previously a field dominated by scientists and engineers. The ‘aesthetic turn’ in data visualization has sparked debates (...) between the new wave and older more scientifically grounded schools of thought on the topic. Sonification is not as well known or commonly practiced as visualization. But sound is a naturally affective, aesthetic and cultural medium. The extension of the aesthetic turn to sonification could transform this field from a scientific curiosity and engineering instrument into a popular mass medium. This paper proposes that a design approach can facilitate an aesthetic turn in sonification that integrates aesthetics and functionality by dissolving divisions between scientific and artistic methods. The first section applies the design perspective to the definition of sonification by replacing the linguistic concept of representation with non-verbal concept of functionality. The next section describes applications of the TaDa design method that raised aesthetic issues particular to sonification practice. The final section proposes a pragmatic aesthetics that distinguishes sonification from the auditory sciences and sonic arts. A design perspective may lead to a future where the general public tunes into pop sonifications for listening enjoyment as well as useful information about the world. (shrink)

continent. 1.4 (2011): 239. In 1998 the Securities and Exchange Commission authorized electronic exchanges. Not only did this give day traders access to buy and sell securities from their desktops, it also made it possible for high powered Wall Street traders to program algorithms to make trades at speeds on the order of milliseconds.(1) The advent of automatic algorithmic trading, now known as high-frequency trading, has vastly accelerated the already increasing speed and volume of trading. This project was an attempt (...) to capture in an ephemeral and environmental way the dizzying acceleration of networked markets. The change in the daily closing value of the Dow and daily trading volume were each set to control a sine wave oscillator between 20 and 800hz. The oscillators were combined to ‘sonify’ the data, creating a changing noise reflecting the daily change and trading volume of the market. The resulting sound is somewhat tedious until the trading volume begins increasing in the 60s and then becomes increasingly complex as high-frequency trading and market volatility set in during the 1990s and 2000s. NOTES 1. Charles Duhigg. “Stock Traders Find Speed Pays, in Milliseconds.” New York Times . July 23, 2009. (shrink)

Numbers have been identified with symbolic data forever. The profound association of both with acoustics, music, and sonic art from Pythagoras to current work is beyond reproach. Recently, sonification looks for ways to realize symbolic data (representing results or measurements) as well as “raw” data (signals, impulses, images, etc.) into compositions. In the strictest sense, everything in a computer is symbolic, that is, represented by 0s and 1s. In the arts, the digital age has broadened and enhanced the conceptual (...) landscape not simply through its servitude to the creative process, but as its partner. However, there is a rich history of the use of data that no doubt has paved the way for many of today’s experiments including my own. (shrink)

Interactive sonification of biomechanical quantities is gaining relevance as a motor learning aid in movement rehabilitation, as well as a monitoring tool. However, existing gaps in sonification research have prevented its widespread recognition and adoption in such applications. The incorporation of embodied principles and musical structures in sonification design has gradually become popular, particularly in applications related to human movement. In this study, we propose a general sonification model for the sit-to-stand transfer, an important activity of (...) daily living. The model contains a fixed component independent of the use-case, which represents the rising motion of the body as an ascending melody using the physical model of a flute. In addition, a flexible component concurrently sonifies STS features of clinical interest in a particular rehabilitative/monitoring situation. Here, we chose to represent shank angular jerk and movement stoppages, through perceptually salient pitch modulations and bell sounds. We outline the details of our technical implementation of the model. We evaluated the model by means of a listening test experiment with 25 healthy participants, who were asked to identify six normal and simulated impaired STS patterns from sonified versions containing various combinations of the constituent mappings of the model. Overall, we found that the participants were able to classify the patterns accurately, confidently, and in a timely manner. The amount of sonified kinematic information significantly impacted classification accuracy. The six STS patterns were also classified with significantly different accuracy depending on their kinematic characteristics. Learning effects were seen in the form of increased accuracy and confidence with repeated exposure to the sound sequences. We found no significant accuracy differences based on the participants' level of music training. Overall, we see our model as a concrete conceptual and technical starting point for STS sonification design catering to rehabilitative and clinical monitoring applications. (shrink)

The number of events and artifacts described as sonification has increased considerably in recent years with some works making a bridge between the representation of data and artistic expression. Flood Tide which sonifies the flow of tidal water is such a work and has achieved a relatively high profile attracting good audiences for its 10 performances to date. It is not entirely obvious however what it is that attracts audiences and whether it is effective at representing the data being (...) sonified. This paper aims to address these issues and is based on a discussion group in which these and other questions are considered. (shrink)

Since sound and music are powerful forces and drivers of human behavior and physiology, we propose the use of sonification to activate healthy breathing patterns in participants to induce relaxation. Sonification is often used in the context of biofeedback as it can represent an informational, non-invasive and real-time stimulus to monitor, motivate or modify human behavior. The first goal of this study is the proposal and evaluation of a distance-based biofeedback system using a tempo- and phase-aligned sonification (...) strategy to adapt breathing patterns and induce states of relaxation. A second goal is the evaluation of several sonification stimuli on 18 participants that were recruited online and of which we analyzed psychometric and behavioral data using, respectively questionnaires and respiration rate and ratio. Sonification stimuli consisted of filtered noise mimicking a breathing sound, nature environmental sounds and a musical phrase. Preliminary results indicated the nature stimulus as most pleasant and as leading to the most prominent decrease of respiration rate. The noise sonification had the most beneficial effect on respiration ratio. While further research is needed to generalize these findings, this study and its methodological underpinnings suggest the potential of the proposed biofeedback system to perform ecologically valid experiments at participants' homes during the COVID-19 pandemic. (shrink)

Le Souffle de la Terre/The Pulse of the Earth is an on-going sound work (1996–20–) by the artist L. Abenavoli. She outlines the sonification methods employed in the making of the work in an analysis of its technical and conceptual features. The work is described in terms of her artistic objectives.

Data sonification is a practice for conducting scientific analysis through the use of sound to represent data. It is now transitioning to a practice for communicating and reaching wider publics by expanding the range of languages and senses for understanding complexity in data-intensive societies. Communicating to wider publics, though, requires that authors intentionally shape sonification in ways that consider the goals and contexts in which publics relate. It requires a specific set of knowledge and skills that design as (...) a discipline could provide. In this article, we interpret five recent sonification projects and locate them on a scale of intentionality in how authors communicate socially relevant issues to publics. (shrink)

The author examines concepts of real time and real-time in relation to notions of perception and processes of sonification. He explores these relationships in three case studies and suggests that sonification can offer a form of reconciliation between ontology and phenomenology, and between ourselves and the flux we are part of.

Shared Musical Lives makes the case for the epistemological and ethical significance of musical experience. Music can be a source of self-knowledge and self-expression, and hence reveal important dimensions of the self to others. This knowledge - of both self and of others - has a moral force as well. Shared musical experience can transform and establish new modes of being with others, cultivate virtues, and expand the moral imagination. The term sonification (which means translating data into non-verbal audible (...) tones) provides an organizing principle for the arguments in the book.Transposing the concept into a philosophical key, this book explores two forms of sonification: first, the process by which musical experience reveals dimensions of the self and relationships with others; and second, philosophical sonification, or the critical examination of philosophical concepts,arguments, and theories in view of what musical experience reveals. These two kinds of sonification are discussed specifically in the context of disability. In this book, author Licia Carlson brings the musical lives of people with cognitive and intellectual disabilities into the foreground in order to challenge and broaden existing conceptions of disability and music and provide new ways of thinking about the philosophies of music and disability. (shrink)

The use of sound to systematically communicate data has been with us for a long time, and has received considerable research, albeit in a broad range of distinct fields of inquiry. Sonification is uniquely capable of conveying series and patterns, trends and outliers…and effortlessly carries affect and emotion related to those data. And sound-either by itself or in conjunction with visual, tactile, or even olfactory representations-can make data exploration more compelling and more accessible to a broader range of individuals. (...) Nevertheless, sonification and auditory displays still occupy only a sliver of popular mindshare: most people have never thought about using non-speech sound in this manner, even though they are certainly very familiar with other intentional uses of sound to convey status, notifications, and warnings. This article provides a brief history of sonification, introduces terms, quickly surveys a range of examples, and discusses the past, present, and as-yet unrealized future promise of using sound to expand the way we can communicate about data, broaden the use of auditory displays in society, and make science more engaging and more accessible. (shrink)

“Walk on the Sun” is an interactive experience of image as music. As explorers move across images that are data projected onto the floor, their movements are visually tracked and used to select pixels in the images which they immediately hear as musical pitches played by various instruments. The sonification design maps color to one of 9 instruments, brightness to one of 50 pitches, and location in the image to panning position, creating 57,600 differentiable musical events. This high resolution (...) and interactive auditory presentation of pixel data enables the blind to explore images of the Sun from the STEREO space mission, nebula and galactic images from Hubble, as well as art masterpieces. Specifically, the blind can hear when hot spots cross the center of the Sun or the solar winds and corona are changing by sonifying virtual geometric structures, such as lines and circles, to create chords of music reflecting the changing content of the selected pixels within that structure as images are played as movies. Originally funded by a NASA/STSCI Ideas grant, the exhibit has toured to more than 12 cities in the US, visiting blind and science centers in the process and receiving enthusiastic response throughout. Plans for additional work furthering NASA wide image sonification standards are in process. (shrink)

As a long‐time scholar of science and art practices, I look particularly at the role of tools and instruments which make these practices possible. I note that science, historically, has favoured visualist imaging, but art, particularly in performance modes, often uses acoustic imaging. Early modern science was dominantly optical in instrumentation, but uses of optics often preceded science use in early modern times. In late modern times, much more complex instrumentation often originated in the sciences, but artists frequently adapted to (...) acoustic practices. Sonifying science, from imaging space phenomena to medical phenomena, is traced here, focusing upon medical sonification in detecting cancers. (shrink)

This article treats the use of sonification in Percy Military Training Hospital’s intensive care unit, through an interview with Anaesthetist Professor Bruno Debien. It starts with a description of the environment completed by some technical information concerning the equipment. This is followed by a commented transcription of the interview with Bruno Debien and concludes with reflections on the nature of audio alarms and their relation to different modes of listening.

Dance is regarded as visual art form by common arts and science perspectives. Definitions of dance as means of communication agree that its message is conveyed by the dancer/choreographer via the human body for the observer, leaving no doubt that dance is performed to be watched. Brain activation elicited by the visual perception of dance has also become a topic of interest in cognitive neuroscience, with regards to action observation in the context of learning, expertise and aesthetics. The view that (...) the aesthetic experience of dance is primarily a visual one is still shared by many artists and cultural institutions, yet there is growing interest in making dance performances accessible for individuals with visual impairment / blindness. Means of supporting the non-visual experience of dance include verbal (audio description), auditive (choreographed body sounds, movement sonification), and haptic (touch tour) techniques, applied for different purposes by artists and researchers, with three main objectives: to strengthen the cultural participation of a non-sighted audience in the cultural and aesthetic experience of dance; to expand the scope of dance as an artistic research laboratory toward novel ways of perceiving what dance can convey; and to inspire new lines of (neuro-cognitive) research beyond watching dance. Reviewing literature from different disciplines and drawing on the personal experience of an inclusive performance of Simon Mayer's “Sons of Sissy,” we argue that a non-exclusively visual approach can be enriching and promising for all three perspectives and conclude by proposing hypotheses for multidisciplinary lines of research. (shrink)

In scientific work, sonification is primarily thought of as a novel way to communicate post hoc research findings to lay audiences but only rarely, if ever, as a component of the research itself. This article argues that, rather than any inherent epistemological limitations of sound as a medium of scientific reasoning, this framing reflects a sociohistorical tendency to “silence” experimental techniques as they become widely adopted—both in terms of the literal silencing of noisy instrumentation and the elision of the (...) role of sound in received discovery accounts. This tendency is well-illustrated by the history of electrophysiology research, particularly the case of the electrical penetration graph, an instrument used by entomologists to study how insects feed on plants. Combining participant observation, interviews, and analysis of EPG technical literature, this article recovers the unacknowledged role of sound in EPG’s exploratory stages and gradual emergence as a standard laboratory technique as well as its gradual silencing as a consequence of this standardizing process. (shrink)

Elephant fish and GPS is an attempt to reflect on data flux, and artistic practice considered as a way to implement an experience specific to a flux. Sonification is particularly well suited to this type of implementation. As such, it leads us to question the nature of this type of experience, the position of the person who is faced with the artistic object, and the position and function of the artist. It allows us to query the status of devices (...) produced by such an artistic practice and the status of what is, traditionally, called a “work of art”. The approach is derived from neither the study of specific artistic projects that could be considered as examples from which a general model could be extrapolated, nor, conversely, that of the enunciation of models illustrated by concrete references. The intention is not to categorize practice or behavior. It is rather an approach that seeks to shift concepts and ways of thinking through progressive analogies aimed at critically questioning the notion of experience in a transition from “object” based logic to “flux” or “field”, based logic, coming from an aesthetic perspective. (shrink)

Language is the original technological prosthesis mediating all transfer of human cognition. The relationships between language, communication and cognition have long been the subject of scientific, philosophic and linguistic inquiry. However, it is through contemporary advancements in neuroscience that we now have unprecedented access to the inner workings of the human brain. Particularly, consumer grade neural scanning technologies like the Muse headset allow non-scientists to view, manipulate and draw conclusions from data generated by their own neural processes. Hence, artists Minka (...) Stoyanova and Lisa Park SoYoung have created an artistic experimental system that utilizes these technologies to address the possibility of communication beyond the confines of linguistic signification. Performing Hypo-Linguistics uses Muse brain sensors to create an abstract audio-visual communication system and feedback loop. Through the visualization and sonification of their respective brain data, these artists perform communication that is both outside of conscious control and beyond sign-based systems of linguistic representation. In this article, we discuss the philosophic and scientific groundwork that supports the creation of this system; our processes of sonification and visualization of the Muse-generated brain data (including the modes by which we actively avoided signification and expected technological aesthetics of data representation); and the outcomes of the performances or the realized interaction between the performers. As deep-learning algorithms become increasingly able to teach computers to mimic human language, it becomes ever more apparent that language is not, in fact, the purview of the human, but only one technological mode invented to facilitate the transfer of information. As with all systems of communication, noise is introduced in this process of transfer. This artistic research seeks to highlight that reality while considering the potentials for alternative forms of technologically mediated communication beyond linguistic signification and below the level of consciousness. (shrink)

Electronic music is advancing not only in the way it is being used in performance but also in the technological sense, due to software developers advancing the ability of the synthesizer to enable the composer to create newer sounds. The introduction of the amino acid and protein synthesizers from MIT is one such example, along with sampling sounds from interstellar bodies through the process of sonification in order to create presets as additional source material for the composer’s palette. The (...) creative process used in creating electronic music on a tablet computer introduces a new musical instrument to be used in live music performances. The fluidity and immediacy of how electronic sounds can be created with tablet computer synthesizers affords the composer to have a new behavioural sense of using them as a musical instrument that can be played intuitively. Exploring this new interface of musical composition is a subject this article will address as well as the psychological aspects pertaining to how an audience can relate to electronic music as an emerging art form removed from the classical music tradition. It will also discuss how the composer of electronic music can affect the listener’s ability to envision new conceptual landscapes, leading to experiencing new ideas and subjective fields of visionary understandings. The composer’s ability to use conceptual models, which influence the way sounds are made and how those sounds influence the listener’s experience, is an important focus of this article. (shrink)

Intelligence augmentation was one of the original goals of computing. Artificial Intelligence (AI) inherits this project and is at the leading edge of computing today. Computing can be considered an extension of brain and body, with mathematical prowess and logic fundamental to the infrastructure of computing. Multimedia computing—sensing, analyzing, and translating data to and from visual images, animation, sound and music, touch and haptics, as well as smell—is based on our human senses and is now commonplace. We use data visualization (...) and sonification, as well as data mining and analysis, to sort through the complexity and vast volume of data coming from the world inside and around us. It helps us ‘see’ in new ways. We can think of this capacity as a new kind of “digital glasses”. The Internet of Living Things (IOLT) is potentially an even more profound extension of ourselves to the world: a network of electronic devices embedded into objects, but now with subcutaneous, ingestible devices, and embedded sensors that include people and other living things. Like the Internet of Things (IOT), living things are connected; we call those connections “ecology”. As the IOT becomes increasingly synonymous with the IOLT, the question of ethics that is at the centre of aesthetics and the arts will move to the forefront of our experience of and regard for the world in and around us. (shrink)

This article investigates the concept of sound, in relation to the new means and sciences from different perspectives, ultimately providing an analysis of the newborn artistic movement of bioart. It is divided into two parts. The first part of the study is based upon reference, investigating the interconnection between art and science. This mechanism is characterized by transformation processes in the interdisciplinary practices that are applied mainly by various artists and movements of the post-Second World War period. The expressive element (...) seeks an unworldly explanation through audio and visual conjunctions. This nature is obvious in Paul Klee’s reflections of musical elements in his paintings, and Rimmington’s attempts to marry audio-visual influences in his ‘colour organs’. The experimentations of composers such as Xenakis and Stockhausen at various locations with light and colour illustrate the continuous quest to render sound by the use of new means. Technology is a vital component of transformation as it enhances syncretic creativity for various art domains such as those that Fluxus deployed. Nam Jun Paik and Dick Higgins introduce radical techniques in their performances as they detach their selves from the parameters that define composition, and use the mind and power of sentiment in order to identify reality aurally and optically. Towards the end of the twentieth century, we witnessed the appearance of new art forms such as bioart. The human body, host of material and immaterial functions, comes to the forefront of art prac¬tice. Its relation to elements such as oscillations and vibrations that express the energy flow is analysed through the model of spiritualism that came from eastern thought. The notion of digital embodiment is presented as a reminder, highlight¬ing the importance of technology in biotechnology and genetics. The second part of the article involves an experiment. This describes how the concept of biomusic is applied with the use of electrocardiography (ECG) data from the MIT PhysioNet database. As sound penetrates the entire human body, it can be analysed in all of its phasma. Using this information, we attempt to translate/transform these biological sound phenomena into music. The sound produced by the elaboration of data that result from biological functions can be described as biomusic. It can be transformed into frequencies related to time and can be expressed in musical themes. Sonification plays an important role in this research as it constitutes a rapid and precise rendering of polymorphic information (in this case the ECG) in musical notes. This modelling and musical attribution leads to two distinguishable results, each concerning different clinical cases (all data belong to a normal heart function and a pathological one). The invention of this novel system is suggested for scientific as well as musical disciplines. It has the ability to be implemented in an experimental form and obtain an educational character. The transformation process avoids compensation throughout the matching process between ECG functions and music, while focusing on the aesthetic factor at the same time. Sound meets art in the world of biomusic as it takes shape through technology, constituting a new medium to further evolve the model of ‘biology into art’ transformation. (shrink)

'So mote it be …' describes a trinity of artworks by Mike Phillips that explore the delicate relationship we have with the extremes of our perception. These data-driven sonifications and visualizations play with the atomic forces that bind the material world as measured through the atomic force microscope (AFM). The works try and locate the trauma of seeing things that are invisible and beyond the resolution of the human eye to a history of cultural experience that similarly tries to see (...) things that might ordinarily have been perceived as occult or at the very least the product of Hollywood. (shrink)