Luthra, Peraza-Santiago, Beeson, Saltzman, Crinnion, and Magnuson (2021) present data from the lexically mediated compensation for coarticulation paradigm that they claim provides conclusive evidence in favor of top-down processing in speech perception. We argue here that this evidence does not support that conclusion. The findings are open to alternative explanations, and we give data in support of one of them (that there is an acoustic confound in the materials). Lexically mediated compensation for coarticulation thus remains elusive, while prior (...) data from the paradigm instead challenge the idea that there is top-down processing in online speech recognition. (shrink)

The locus equation proposal ignores a fundamental difference between human speech perception and nonhuman echolocation and sound localization, offers a questionable account of the function of consonant-vowel coarticulation, and is further undermined if the effects of other forms of coarticulation are considered. The function of coarticulation is to convey phonetic information rapidly and reliably.

A long-standing question in cognitive science is how high-level knowledge is integrated with sensory input. For example, listeners can leverage lexical knowledge to interpret an ambiguous speech sound, but do such effects reflect direct top-down influences on perception or merely postperceptual biases? A critical test case in the domain of spoken word recognition is lexically mediated compensation for coarticulation (LCfC). Previous LCfC studies have shown that a lexically restored context phoneme (e.g., /s/ in Christma#) can alter the perceived place (...) of articulation of a subsequent target phoneme (e.g., the initial phoneme of a stimulus from a tapes-capes continuum), consistent with the influence of an unambiguous context phoneme in the same position. Because this phoneme-to-phoneme compensation for coarticulation is considered sublexical, scientists agree that evidence for LCfC would constitute strong support for top–down interaction. However, results from previous LCfC studies have been inconsistent, and positive effects have often been small. Here, we conducted extensive piloting of stimuli prior to testing for LCfC. Specifically, we ensured that context items elicited robust phoneme restoration (e.g., that the final phoneme of Christma# was reliably identified as /s/) and that unambiguous context-final segments (e.g., a clear /s/ at the end of Christmas) drove reliable compensation for coarticulation for a subsequent target phoneme. We observed robust LCfC in a well-powered, preregistered experiment with these pretested items (N = 40) as well as in a direct replication study (N = 40). These results provide strong evidence in favor of computational models of spoken word recognition that include top–down feedback. (shrink)

The question of when and how bottom‐up input is integrated with top‐down knowledge has been debated extensively within cognition and perception, and particularly within language processing. A long running debate about the architecture of the spoken‐word recognition system has centered on the locus of lexical effects on phonemic processing: does lexical knowledge influence phoneme perception through feedback, or post‐perceptually in a purely feedforward system? Elman and McClelland (1988) reported that lexically restored ambiguous phonemes influenced the perception of the following phoneme, (...) supporting models with feedback from lexical to phonemic representations. Subsequently, several authors have argued that these results can be fully accounted for by diphone transitional probabilities in a feedforward system (Cairns et al., 1995; Pitt & McQueen, 1998). We report results strongly favoring the original lexical feedback explanation: lexical effects were present even when transitional probability biases were opposite to those of lexical biases. (shrink)

A number of studies have shown that phonetic peculiarities, especially at the coarticulation level, exist in the disfluent as well as in the perceptively fluent speech of people who stutter (PWS). However, results from fluent speech are disparate and not easily interpretable. Are the coarticulatory features a manifestation of the disorder, or rather a compensation for the disorder itself? Our purpose is to investigate the coarticulatory behavior in the fluent speech of PWS in the attempt to answer the question (...) on its symptomatic or adaptive nature. In order to achieve this, we have studied the speech of 21 adult PWS (10 French and 11 Italian) compared to that of 20 fluent adults (10 French and 10 Italian). The participants had to repeat simple CV syllables in short carrier sentences, where C = /b, d, g/ and V = /a, i, u/. Crucially, this repetition task was performed under a condition with normal auditory feedback (NAF) for both the groups, and under a fluency-enhancing condition (only for PWS) due to an altered auditory feedback (AAF). The degree of coarticulation was measured by means of the Locus Equations (LE). Under NAF conditions, the coarticulation degree observed in fluent PWS speech is lower than that of the PWNS, and, more importantly, in AAF condition, PWS coarticulation appears even weaker than in the NAF condition. The latter result allows to interpret the low degree of coarticulation found in fluent speech of PWS under NAF condition as a compensation for the disorder, based on the fact that coarticulation is weaker in fluency-enhancing conditions. Since a lower degree of coarticulation is associated to a greater separation between the places of articulation of the consonant and the vowel, these results are compatible with the hypothesis that larger articulatory movements could be responsible for the stabilization of the PWS speech motor system, increasing the kinesthetic feedback from the effector system. This interpretation shares with a number of relatively recent proposal the idea that stuttering derives from an impaired feedforward (open-loop) control system, which makes PWS rely more heavily on a feedback-based (closed loop) motor control strategy. (shrink)

Musical performance is a multimodal experience, for performers and listeners alike. This paper reports on a pilot study which constitutes the first step toward a comprehensive approach to the experience of music as performed. We aim at bridging the gap between qualitative and quantitative approaches, by combining methods for data collection. The purpose is to build a data corpus containing multimodal measures linked to high-level subjective observations. This will allow for a systematic inclusion of the knowledge of music professionals in (...) an analytic framework, which synthesizes methods across established research disciplines. We outline the methods we are currently developing for the creation of a multimodal data corpus dedicated to the analysis and exploration of instrumental music performance from the perspective of embodied music cognition. This will enable the study of the multiple facets of instrumental music performance in great detail, as well as lead to the development of music creation techniques that take advantage of the cross-modal relationships and higher-level qualities emerging from the analysis of this multi-layered, multimodal corpus. The results of the pilot project suggest that qualitative analysis through stimulated recall is an efficient method for generating higher-level understandings of musical performance. Furthermore, the results indicate several directions for further development, regarding observational movement analysis, and computational analysis of coarticulation, chunking, and movement qualities in musical performance. We argue that the development of methods for combining qualitative and quantitative data are required to fully understand expressive musical performance, especially in a broader scenario in which arts, humanities, and science are increasingly entangled. The future work in the project will therefore entail an increasingly multimodal analysis, aiming to become as holistic as is music in performance. (shrink)

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of early (...) cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing. (shrink)

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of early (...) cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing. (shrink)

Virtuosity in music performance is often associated with fast, precise, and efficient sound-producing movements. The generation of such highly skilled movements involves complex joint and muscle control by the central nervous system, and depends on the ability to anticipate, segment, and coarticulate motor elements, all within the biomechanical constraints of the human body. When successful, such motor skill should lead to what we characterize as fluency in musical performance. Detecting typical features of fluency could be very useful for technology-enhanced learning (...) systems, assisting and supporting students during their individual practice sessions by giving feedback and helping them to adopt sustainable movement patterns. In this study, we propose to assess fluency in musical performance as the ability to smoothly and efficiently coordinate while accurately performing slow, transitionary, and rapid movements. To this end, the movements of three cello players and three drummers at different levels of skill were recorded with an optical motion capture system, while a wireless electromyogrphy (EMG) system recorded the corresponding muscle activity from relevant landmarks. We analyze the kinematic and coarticulation characteristics of these recordings separately and then propose a combined model of fluency in musical performance predicting music sophistication. Results suggest movements from expert performers' are characterized by consistently smooth strokes and scaling of muscle phasic coactivation. The explored model of fluency as function of movement smoothness and coarticulation patterns was shown to be limited by the sample size but serves as a proof of concept. Results from this study show the potential of a technology-enhanced objective measure of fluency in musical performance, which could lead to improved practices for aspiring musicians, instructors, and researchers. (shrink)

This article reports results from a program that produces high‐quality animation of facial expressions and head movements as automatically as possible in conjunction with meaning‐based speech synthesis, including spoken intonation. The goal of the research is as much to test and define our theories of the formal semantics for such gestures, as to produce convincing animation. Towards this end, we have produced a high‐level programming language for three‐dimensional (3‐D) animation of facial expressions. We have been concerned primarily with expressions conveying (...) information correlated with the intonation of the voice: This includes the differences of timing, pitch, and emphasis that are related to such semantic distinctions of discourse as “focus,”“topic,” and “comment,”“theme” and “rheme,” or “given” and “new” information. We are also interested in the relation of affect or emotion to facial expression. Until now, systems have not embodied such rule‐governed translation from spoken utterance meaning to facial expressions. Our system embodies rules that describe and coordinate these relations: intonation/information, intonation/affect, and facial expressions/affect. A meaning representation includes discourse information: What is contrastive/background information in the given context, and what is the “topic” or “theme” of the discourse? The system maps the meaning representation into how accents and their placement are chosen, how they are conveyed over facial expression, and how speech and facial expressions are coordinated. This determines a sequence of functional groups: lip shapes, conversational signals, punctuators, regulators, and manipulators. Our algorithms then impose synchrony, create coarticulation effects, and determine affectual signals, eye and head movements. The lowest level representation is the Facial Action Coding System (FACS), which makes the generation system portable to other facial models. (shrink)

Cet article propose un tour d’horizon de différents types de biais que l’on peut rencontrer dans les études basées sur l’annotation de corpus vidéo de langue des signes. En tâchant de situer objectivement les choix effectués et les biais potentiels à chaque étape, nous décrivons les méthodologies que nous avons mises en place dans trois études, portant respectivement sur la synchronisation des composantes corporelles, le mouvement des sourcils et les clignements des yeux, puis la coarticulation.

Buffed up to a metallic shine; loose fitting, lopsided, or kludgy; getting in the way or getting lost; collapsing in an explosion of dust caught on the warehouse CCTV. Modern things are going their own ways, and this book attempts to follow them. A course of thought about their comings and goings and cascading side effects, _The Way Things Go_ offers a thesis demonstrated via a century-long countdown of stuff. Modernist critical theory and aesthetic method, it argues, are bound up (...) with the inhuman fate of things as novelty becoming waste. Things are seldom at rest. Far more often they are going their own ways, entering and exiting our zones of attention, interest, and affection. Aaron Jaffe is concerned less with a humanist story of such things—offering anthropomorphizing narratives about recouping the items we use—as he is with the seemingly inscrutable, inhuman capacities of things for coarticulation and coherence. He examines the tension between this inscrutability on the one hand, and the ways things seem ready-made for understanding on the other hand, by means of exposition, thing-and-word-play, conceptual art, essayism, autopoesis, and prop comedy. Among other novelties and detritus, T_he Way Things Go_ delves into books, can openers, roller skates, fat, felt, soap, joy buzzers, hobbyhorses, felt erasers, sleds, magic rabbits, and urinals. But it stands apart from the recent flood of thing-talk, rebuking the romantic tendencies caught up in the pathetic nature of debris defining the conversation. Jaffe demonstrates that literary criticism is the one mode of analysis that can unpack the many things that, at first glance, seem so nonliterary. (shrink)

This article draws from Hannah Arendt's theory of “inter-est” to formulate a model of coalition discourse that can coarticulate difference and commonality and approach them as mutually nourishing conditions rather than as polarities. By disrupting the normative fantasies of unified, a priori subjectivity and universal truth, interest-based discourse facilitates political interactions that neither rely on sameness nor reify difference to the exclusion of connection.

This article draws from Hannah Arendt's theory of “inter-est” to formulate a model of coalition discourse that can coarticulate difference and commonality and approach them as mutually nourishing conditions rather than as polarities. By disrupting the normative fantasies of unified, a priori subjectivity and universal truth, interest-based discourse facilitates political interactions that neither rely on sameness nor reify difference to the exclusion of connection.

Using agent‐based modelling, Harrington, Kleber, Reubold, Schiel & Stevens (2018) develop a unified model of sound change based on cognitive processing of human speech and theories of how social factors constrain the spread of change throughout a community. They conclude that many types of change result from how biases in the phonetic distribution of phonological categories are transmitted via accommodation processes between individuals in interaction.

In lieu of an abstract, here is a brief excerpt of the content:Exception in Žižek’s ThoughtErik Vogt (bio)One cannot fail to be struck by the repeated occurrences and invocations of some logic of exception as well as by the proliferation of examples or stand-ins for exceptional positions (“Jew”; “woman”; “class struggle”) or exceptional collectives (“proletariat”; “slum dwellers”) in many of Slavoj Žižek’s writings. The significance of thinking exception is evident not only in Žižek’s powerful reconceptualization of (a supposedly outdated) ideology (...) critique and in his intriguing materialist account of the relationship between Judaism and Christianity, but also in his interventions into contemporary debates as to what constitutes proper politics. For this reason, it might be appropriate, in a first sketch, to attempt to weave together some of the threads that combine into Žižek’s (critique of the) logic of exception.11The Symptom and Its Logic of ExceptionAlready Žižek’s early brilliant elaboration of a contemporary ideology critique in terms of a powerful coarticulation of Marx and Freud analyzes the “social symptom” in light of its hidden logic of exception [Sublime Object 11–53]. Since the “social symptom” is not to be grasped as some imbalance external to universality, but rather as its constitutive moment, it marks “a point of breakdown heterogeneous” to a given universal field that at the same time is “necessary for that field to achieve its closure, its accomplished form” [21]. A symptomatic ideology critique will seize on the logic of exception implicit in the fissures of universality by deciphering its “falseness,” insofar “as it necessarily includes a specific case which breaks its unity, lays open its falsity” [21]. What is impossible, then, is the claim to some totality or universality without including in it “the point of exception functioning as its internal negation” [23]. This seems to suggest that the “social symptom” does not present any obstacles to its ideological-critical decipherment according to the logic of exception. However, Žižek makes clear immediately that it often does not effect the dissolution of the “social symptom”; on the contrary, it might even reinforce it, insofar as this interpretive procedure appeals to and relies on an addressee that continues to be figured in terms of consistency, completeness, totality. Consequently, this interpretive procedure of dissolving the symptom remains stuck in the very logic of exception that it attempts to unearth [73–74]. This is the reason why Žižek suggests supplementing this notion of the “social symptom” as ciphered message with the notion of fantasy. He describes the relation between symptom and fantasy in the following manner: “Symptom implies and addresses some non-barred, consistent big Other which will retroactively confer on it its meaning; fantasy implies a crossed-out, barred, non-whole, inconsistent [End Page 61] Other—that is to say, it is filling out a void in the Other” [74]. A contemporary ideology critique will therefore have to account for both the exceptional status of the social symptom and its underlying social fantasy. In other words, ideology critiques must consist of two stages, of two complementary procedures:—one is discursive, the “symptomal reading” of the ideological text [...] demonstrating how a given ideological field is a result of a montage of heterogeneous “floating signifiers,” of their totalization through the intervention of certain “nodal points”;—the other aims at extracting the kernel of enjoyment, at articulating the way in which [...] an ideology implies, manipulates, produces a pre-ideological enjoyment structured in fantasy.[125]In short, what is required from ideology critique is an interpretation of the social symptom in such a way that it brings to light the very fundamental fantasy that, precisely as a preideological blockage to interpretation, has then to be traversed and correlated to the identification with the symptom. The social symptom has to be grasped as master signifier and as objet a. And Žižek illustrates the working of (a theory of) ideology via anti-Semitism and its construction of the “Jew.”2Briefly, anti-Semitic ideology articulates the “Jew” through a network of symbolic “overdetermination”; that is, the “Jew” becomes the point of condensation for heterogeneous and often incompatible attributes. What has to be taken into account, moreover, is the inversion... (shrink)

In this contribution we will approach language and images in different modalities: speech and face, anticipated and imagined movements, illusions on the sound by the image. It will be the opportunity for us to revisit the Gestalt concepts which were considered obsolete since structuralism in Humanities. As instantiated by Gilbert Durand in The Anthropological Structures of the Imaginary (1999, French 1st ed. 1960), we shall recall that Gestalt is not—even implicitly—an exclusively static approach to cognition. On the contrary we will (...) emphasize that forms can emerge from movements and stabilize in memory, thence morphology. And for speech, we will consider phonology as the stabilized outcome from audible and visible gestures of the mouth—caught in the time-course of a flow of some four in six syllables by second—via the perception of motor coordinations: which is necessary to master the most common expression of language between humans. We will adopt here also for the perceptual flows of spoken language, Multimodal Scene Analysis, the ongoing legacy of Gestalt Theory, including one of our main research topics: the gestural expression of the face and the hand for Deaf people, who practise as an augment French LPC (Langue française Parlée Complétée), adapted in 1967 from Cued Speech by its originator Dr Richard Cornett. (shrink)