Facial expressions are a vital way for humans to show their perceived emotions. It is convenient for detecting and recognizing expressions or micro-expressions by annotating a lot of data in deep learning. However, the study of video-based expressions or micro-expressions requires that coders have professional knowledge and be familiar with action unit coding, leading to considerable difficulties. This paper aims to alleviate this situation. We deconstruct facial muscle movements from the motor cortex and systematically sort out the relationship (...) among facial muscles, AU, and emotion to make more people understand coding from the basic principles: We derived the relationship between AU and emotion based on a data-driven analysis of 5,000 images from the RAF-AU database, along with the experience of professional coders.We discussed the complex facial motor cortical network system that generates facial movement properties, detailing the facial nucleus and the motor system associated with facial expressions.The supporting physiological theory for AU labeling of emotions is obtained by adding facial muscle movements patterns.We present the detailed process of emotion labeling and the detection and recognition of AU.Based on the above research, the video's coding of spontaneous expressions and micro-expressions is concluded and prospected. (shrink)

That facial expressions are universal emotion signals has been supported by observers agreeing on the emotion mimed by actors. We show that actors can mime a diverse range of states: emotions, cognitions, physical states, and actions. English, Hindi, and Malayalam speakers viewed 25 video clips and indicated the state conveyed. Within each language, at least 23 of the 25 clips were recognised above chance and base rate. Facial expressions of emotions are not special in their recognisability, and it (...) is miming that may be the universal human ability. (shrink)

This article aims to determine to what extent photographic practices in psychology, psychiatry and physiology contributed to the definition of the external bodily signs of passions and emotions in the second half of the 19th century in France. Bridging the gap between recent research in the history of emotions and photographic history, the following analyses focus on the photographic production of scientists and photographers who made significant contributions to the study of expressions and gestures, namely Duchenne de Boulogne, Charles Darwin, (...) Paul Richer and Albert Londe. This article argues that photography became a key technology in their works due to the adequateness of the exposure time of different cameras to the duration of the bodily manifestations to be recorded, and that these uses constituted facial expressions and bodily gestures as particular objects for the scientific study. (shrink)

Following Yik and Russell (1999) a judgement paradigm was used to examine to what extent differential accuracy of recognition of facial expressions allows evaluation of the well-foundedness of different theoretical views on emotional expression. Observers judged photos showing facial expressions of seven emotions on the basis of: (1) discrete emotion categories; (2) social message types; (3) appraisal results; or (4) action tendencies, and rated their confidence in making choices. Emotion categories and appraisals were judged significantly more accurately and (...) confidently than messages or action tendencies. These results do not support claims of primacy for message or action tendency views of facial expression. Based on a componential model of emotion it is suggested that judges can infer components from categories and vice versa. (shrink)

In this article, we review empirical evidence regarding the relationship between facial expression and emotion during infancy. We focus on differential emotions theory’s view of this relationship because of its theoretical and methodological prominence. We conclude that current evidence fails to support its proposal regarding a set of pre-specified facial expressions that invariably reflect a corresponding set of discrete emotions in infants. Instead, the relationship between facial expression and emotion appears to be more complex. Some facial (...) expressions may have different meanings in infants than in children and adults. In addition, nonemotion factors may sometimes lead to the production of “emotional” facial expressions. We consider alternative perspectives on the nature of emotion and emotional expression in infancy with particular focus on differentiation and dynamical systems approaches. (shrink)

ABSTRACTUsing faces as the priming stimuli, the present study explored the influence of facial expressions on the activation of gender stereotypes using a lexical decision paradigm. Experiment 1 explored the activation of gender stereotypes when the facial primes contained only gender information. The results showed that gender stereotypes were activated. In Experiment 2, the facial primes contained both gender category and expression information. The results indicated that gender stereotypes were not activated. Experiment 3 required the participants to (...) make emotion, gender, or impression decisions concerning the facial primes before the lexical decision task. The results showed that gender stereotypes were not activated in emotion and impression decisions conditions, whereas stereotypes were activated in gender decisions condition. These finding suggest that facial expressions can inhibit automatic activation of gender stereotypes, unless the perceivers perform gender categorization processing to prime f... (shrink)

With a few yet increasing number of exceptions, the cognitive sciences enthusiastically endorsed the idea that there are basic facial expressions of emotions that are created by specific configurations of facial muscles. We review evidence that suggests an inherent role for context in emotion perception. Context does not merely change emotion perception at the edges; it leads to radical categorical changes. The reviewed findings suggest that configurations of facial muscles are inherently ambiguous, and they call for a (...) different approach towards the understanding of facial expressions of emotions. Prices of sticking with the modal view, and advantages of an expanded view, are succinctly reviewed. (shrink)

According to a common sense theory, facial expressions signal specific emotions to people of all ages and therefore provide children easy access to the emotions of those around them. The evidence, however, does not support that account. Instead, children’s understanding of facial expressions is poor and changes qualitatively and slowly over the course of development. Initially, children divide facial expressions into two simple categories (feels good, feels bad). These broad categories are then gradually differentiated until an adult (...) system of discrete categories is achieved, likely in the teen years. Children’s understanding of most specific emotions begins not with facial expressions, but with their understanding of the emotion’s antecedents and behavioral consequences. (shrink)

This article reports results from a program that produces high‐quality animation of facial expressions and head movements as automatically as possible in conjunction with meaning‐based speech synthesis, including spoken intonation. The goal of the research is as much to test and define our theories of the formal semantics for such gestures, as to produce convincing animation. Towards this end, we have produced a high‐level programming language for three‐dimensional (3‐D) animation of facial expressions. We have been concerned primarily with (...) expressions conveying information correlated with the intonation of the voice: This includes the differences of timing, pitch, and emphasis that are related to such semantic distinctions of discourse as “focus,”“topic,” and “comment,”“theme” and “rheme,” or “given” and “new” information. We are also interested in the relation of affect or emotion to facial expression. Until now, systems have not embodied such rule‐governed translation from spoken utterance meaning to facial expressions. Our system embodies rules that describe and coordinate these relations: intonation/information, intonation/affect, and facial expressions/affect. A meaning representation includes discourse information: What is contrastive/background information in the given context, and what is the “topic” or “theme” of the discourse? The system maps the meaning representation into how accents and their placement are chosen, how they are conveyed over facial expression, and how speech and facial expressions are coordinated. This determines a sequence of functional groups: lip shapes, conversational signals, punctuators, regulators, and manipulators. Our algorithms then impose synchrony, create coarticulation effects, and determine affectual signals, eye and head movements. The lowest level representation is the Facial Action Coding System (FACS), which makes the generation system portable to other facial models. (shrink)

In two experiments, we tested whether fearful facial expressions capture attention in an awareness-independent fashion. In Experiment 1, participants searched for a visible neutral face presented at one of two positions. Prior to the target, a backward-masked and, thus, invisible emotional or neutral face was presented as a cue, either at target position or away from the target position. If negative emotional faces capture attention in a stimulus-driven way, we would have expected a cueing effect: better performance where fearful (...) or disgusted facial cues were presented at target position than away from the target. However, no evidence of capture of attention was found, neither in behavior, nor in event-related lateralizations. In Experiment 2, we went one step further and used fearful faces as visible targets, too. Thereby, we sought to boost awareness-independent capture of attention by fearful faces. However, still, we found no significant attention-capture effect. Our results show that fearful facial expressions do not capture attention in an awareness-independent way. Results are discussed in light of existing theories. (shrink)

The aim of this review is to show the fruitfulness of using images of facial expressions as experimental stimuli in order to study how neural systems support biologically relevant learning as it relates to social interactions. Here we consider facial expressions as naturally conditioned stimuli which, when presented in experimental paradigms, evoke activation in amygdala–prefrontal neural circuits that serve to decipher the predictive meaning of the expressions. Facial expressions offer a relatively innocuous strategy with which to investigate (...) these normal variations in affective information processing, as well as the promise of elucidating what role the aberrance of such processing might play in emotional disorders. (shrink)

Facial expressions of emotion play an important role in human social interactions. However, posed expressions of emotion are not always the same as genuine feelings. Recent research has found that facial expressions are increasingly used as a tool for understanding social interactions instead of personal emotions. Therefore, the credibility assessment of facial expressions, namely, the discrimination of genuine (spontaneous) expressions from posed (deliberate/volitional/deceptive) ones, is a crucial yet challenging task in facial expression understanding. With recent advances (...) in computer vision and machine learning techniques, rapid progress has been made in recent years for automatic detection of genuine and posed facial expressions. This paper presents a general review of the relevant research, including several spontaneous vs. posed (SVP) facial expression databases and various computer vision based detection methods. In addition, a variety of factors that will influence the performance of SVP detection methods are discussed along with open issues and technical challenges in this nascent field. (shrink)

We discuss four methodological issues regarding cross-cultural judgment studies of facial expressions of emotion involving design, sampling, stimuli, and dependent variables. We use examples of relatively recent studies in this area to highlight and discuss these issues. We contend that careful consideration of these, and other, cross-cultural methodological issues can help researchers minimize methodological errors, and can guide the field to address new and different research questions that can continue to facilitate an evolution in the field’s thinking about the (...) nature of culture, emotion, and facial expressions. (shrink)

Deficits in recognition of facial expressions of emotions are considered to be an important factor explaining impairments in social functioning and affective reactions of schizophrenic patients. Many studies confirmed such deficits while controversies remained concerning the emotion valence and modality. The aim of the study was to explore the process of recognizing facial expressions of emotion in the group of schizophrenic patients by analyzing the role of emotion valence, modality and gender of the model. Results of the group (...) of 35 patients and 35 matched controls indicate that while schizophrenic patients show general impairment in recognizing facial expressions of both positive and the majority of negative emotions, there are differences in deficits for particular emotions. Expressions also appeared to be more ambiguous for the patients while variables connected with gender were found less significant. (shrink)

A substantial body of research has established that even when we are not consciously aware of the faces of others we are nevertheless sensitive to, and impacted by their facial expression. In this paper, we consider this body of research from a new perspective by examining the functions of unconscious perception revealed by these studies. A consideration of the literature from this perspective highlights that existing research methods are limited when it comes to revealing possible functions of unconscious perception. (...) The critical shortcoming is that in all of the methods, the perceived facial expression remains outside of awareness. This is a problem because there are good reasons to believe that one important function of unconsciously perceived negative faces is to attract attention so that they are consciously perceived; such conscious perception, however, is never allowed with existing methodologies. We discuss recent studies of emotional face perception under conditions of visual search that address this issue directly. Further, we suggest that methodologies that do not examine cognitive processes as they occur in more natural settings may result in fundamental misunderstandings of human cognition. (shrink)

Although cognitive science was multidisciplinary from the start, an under-emphasis on anthropology has left the field with limited research in small scale, indigenous societies. Neglecting the anthropological perspective is risky, given that once-canonical cognitive science findings have often been shown to be artifacts of enculturation rather than cognitive universals. This imbalance has become more problematic as the increased use of Western theory-driven approaches, many of which assume human uniformity (“universality”), confronts the absence of a robust descriptive base that might provide (...) clarifying or even contrary evidence. We highlight the need for remedies to such shortcomings by suggesting a two-fold methodological shift. First, studies conducted in indigenous societies can benefit by relying on multidisciplinary research groups to diminish ethnocentrism and enhance the quality of the data. Second, studies devised for Western societies can readily be adapted to the changing settings encountered in the field. Here, we provide examples, drawn from the areas of emotion and facial expressions, to illustrate potential solutions to recurrent problems in enhancing the quality of data collection, hypothesis testing, and the interpretation of results. (shrink)

The current meta-analysis reviews 24 studies on self-reported emotional reactions to facial expressions in socially anxious versus nonanxious individuals. We hypothesized that socially anxious individuals would perceive all face types as less approachable, more negative, and more arousing. After correcting for biases, results showed that socially anxious individuals, compared to controls, reported lower approachability to all types of expressions and higher arousal in response to neutral expressions. Variances among effects usually could not be explained by the proposed moderators. This (...) suggests that current conceptualizations of social anxiety should take into account the willingness to approach social stimuli rather than global measures of emotion or arousal. (shrink)

Emotional processing without conscious awareness plays an important role in human social interaction. Several behavioral studies reported that subliminal presentation of photographs of emotional facial expressions induces unconscious emotional processing. However, it was difficult to elicit strong and robust effects using this method. We hypothesized that dynamic presentations of facial expressions would enhance subliminal emotional effects and tested this hypothesis with two experiments. Fearful or happy facial expressions were presented dynamically or statically in either the left or (...) the right visual field for 20 (Experiment 1) and 30 (Experiment 2) ms. Nonsense target ideographs were then presented, and participants reported their preference for them. The results consistently showed that dynamic presentations of emotional facial expressions induced more evident emotional biases toward subsequent targets than did static ones. These results indicate that dynamic presentations of emotional facial expressions induce more evident unconscious emotional processing. (shrink)

In children experiencing pain, the study of the social context of facial expressions might help to evaluate evolutionary and conditioning hypotheses of behavioural development. Social motivations and influences may be complex, as seen in studies of children having their ears pierced, and in studies of everyday pain in children. A study of opposing predictions of the long-term effects of parental caregiving is suggested.

ABSTRACTResponses to surprising events are dynamic. We argue that initial responses are primarily driven by the unexpectedness of the surprising event and reflect an interrupted and surprised state in which the outcome does not make sense yet. Later responses, after sense-making, are more likely to incorporate the valence of the outcome itself. To identify initial and later responses to surprising stimuli, we conducted two repetition-change studies and coded the general valence of facial expressions using computerised facial coding and (...) specific facial action using the Facial Action Coding System. Results partly supported our unfolding logic. The computerised coding showed that initial expressions to positive surprises were less positive than later expressions. Moreover, expressions to positive and negative surprises were initially similar, but after some time differentiated depending on the valence of the event. Importantly, these patterns were particularly pronounced in a subset of facially expres... (shrink)

We report two experimental studies of human perceptions of robotic facial expressions while systematically varying context effects and the cultural background of subjects (n = 93). Except for Fear, East Asian and Western subjects were not significantly different in recognition rates, and, while Westerners were better at judging affect from mouth movement alone, East Asians were not any better at judging affect based on eye/brow movement alone. Moreover, context effects appeared capable of over-riding such cultural differences, most notably for (...) Fear. The results seem to run counter to previous theories of cultural differences in facial expression based on emoticons and eye fixation patterns. We connect this to broader research in cognitive science – suggesting the findings support a dynamical systems view of social cognition as an emergent phenomenon. The results here suggest that, if we can induce appropriate context effects, it may be possible to createculture-neutral modelsof robots and affective interaction. (shrink)

We report two experimental studies of human perceptions of robotic facial expressions while systematically varying context effects and the cultural background of subjects (n = 93). Except for Fear, East Asian and Western subjects were not significantly different in recognition rates, and, while Westerners were better at judging affect from mouth movement alone, East Asians were not any better at judging affect based on eye/brow movement alone. Moreover, context effects appeared capable of over-riding such cultural differences, most notably for (...) Fear. The results seem to run counter to previous theories of cultural differences in facial expression based on emoticons and eye fixation patterns. We connect this to broader research in cognitive science – suggesting the findings support a dynamical systems view of social cognition as an emergent phenomenon. The results here suggest that, if we can induce appropriate context effects, it may be possible to createculture-neutral modelsof robots and affective interaction. (shrink)