In this article, we review empirical evidence regarding the relationship between facial expression and emotion during infancy. We focus on differential emotions theory’s view of this relationship because of its theoretical and methodological prominence. We conclude that current evidence fails to support its proposal regarding a set of pre-specified facial expressions that invariably reflect a corresponding set of discrete emotions in infants. Instead, the relationship between facial expression and emotion appears to be more complex. Some (...) class='Hi'>facial expressions may have different meanings in infants than in children and adults. In addition, nonemotion factors may sometimes lead to the production of “emotional” facial expressions. We consider alternative perspectives on the nature of emotion and emotional expression in infancy with particular focus on differentiation and dynamical systems approaches. (shrink)

Background: This study developed a photo and video database of 4-to-6-year-olds expressing the seven induced and posed universal emotions and a neutral expression. Children participated in photo and video sessions designed to elicit the emotions, and the resulting images were further assessed by independent judges in two rounds. Methods: In the first round, two independent judges, experts in the Facial Action Coding System, firstly analysed 3,668 emotions facial expressions stimuli from 132 children. Both judges reached 100% agreement (...) regarding 1,985 stimuli, which were then selected for a second round of analysis between judges 3 and 4. Results: The result was 1,985 stimuli were produced from 124 participants. A Kappa index of 0.70 and an accuracy of 73% between experts were observed. Lower accuracy was found for emotional expression by 4-year-olds than 6-year-olds. Happiness, disgust and contempt had the highest agreement. After a sub-analysis evaluation of all four judges, 100% agreement was reached for 1,381 stimuli which compound the ChildEFES database with 124 participants and 51% induced photographs. The number of stimuli of each emotion were: 87 for neutrality, 363 for happiness, 170 for disgust, 104 for surprise, 152 for fear, 144 for sadness, 157 for anger 157, and 183 for contempt. Conclusions: The findings show that this photo and video database can facilitate research on the mechanisms involved in early childhood recognition of facial emotions in children, contributing to the understanding of facial emotion recognition deficits which characterise several neurodevelopmental and psychiatric disorders. (shrink)

ABSTRACTAdults perceive emotional expressions categorically, with discrimination being faster and more accurate between expressions from different emotion categories than between two stimuli from the same category. The current study sought to test whether facial expressions of happiness and fear are perceived categorically by pre-verbal infants, using a new stimulus set that was shown to yield categorical perception in adult observers. These stimuli were then used with 7-month-old infants using a habituation and visual preference paradigm. Infants (...) were first habituated to an expression of one emotion, then presented with the same expression paired with a novel expression either from the same emotion category or from a different emotion category. After habituation to fear, infants displayed a novelty preference for pairs of between-category expressions, but n... (shrink)

ABSTRACTThe recognition of emotional facial expressions is often subject to contextual influence, particularly when the face and the context convey similar emotions. We investigated whether spontaneous, incidental affective theory of mind inferences made while reading vignettes describing social situations would produce context effects on the identification of same-valenced emotions as well as differently-valenced emotions conveyed by subsequently presented faces. Crucially, we found an effect of context on reaction times in both experiments while, in line with previous work, (...) we found evidence for a context effect on accuracy only in Experiment 1. This demonstrates that affective theory of mind inferences made at the pragmatic level of a text can automatically, contextually influence the perceptual processing of emotional facial expressions in a separate task even when those emotions are of a distinctive valence. Thus, our novel findings suggest that language acts as a contex... (shrink)

Research on facial emotion expression has mostly focused on emotion recognition, assuming that a small number of discrete emotions is elicited and expressed via prototypical facial muscle configurations as captured in still photographs. These are expected to be recognized by observers, presumably via template matching. In contrast, appraisal theories of emotion propose a more dynamic approach, suggesting that specific elements of facial expressions are directly produced by the result of certain appraisals and predicting the facial (...) patterns to be expected for certain appraisal configurations. This approach has recently been extended to emotion perception, claiming that observers first infer individual appraisals and only then make categorical emotion judgments based on the estimated appraisal patterns, using semantic inference rules. Here we report two related studies to empirically investigate the facial action unit configurations that are used by actors to convey specific emotions in short affect bursts and to examine to what extent observers can infer a person’s emotions from the predicted facial expression configurations. The results show that 1) professional actors use many of the predicted facial action unit patterns to enact systematically specified appraisal outcomes in a realistic scenario setting, and 2) naïve observers infer the respective emotions based on highly similar facial movement configurations with a degree of accuracy comparable to earlier research findings. Based on estimates of underlying appraisal criteria for the different emotions we conclude that the patterns of facial action units identified in this research correspond largely to prior predictions and encourage further research on appraisal-driven expression and inference. (shrink)

Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition. Based on machine learning, such technology can tremendously increase the amount of processed data. FER is now easily accessible and has been validated for the classification of standardized prototypical facial expressions. However, applicability to more naturalistic facial expressions still remains uncertain. Hence, we test and compare performance of three (...) different FER systems with human emotion recognition for standardized posed facial expressions and for non-standardized acted facial expressions. For the standardized images, all three systems classify basic emotions accurately and they are mostly on par with human raters. For the non-standardized stimuli, performance drops remarkably for all three systems, but Azure still performs similarly to humans. In addition, all systems and humans alike tend to misclassify some of the non-standardized emotional facial expressions as neutral. In sum, emotion recognition by automated facial expression recognition can be an attractive alternative to human emotion recognition for standardized and non-standardized emotional facial expressions. However, we also found limitations in accuracy for specific facial expressions; clearly there is need for thorough empirical evaluation to guide future developments in computer vision of emotional facial expressions. (shrink)

One facet of Vigil's socio-relational framework of expressive behaviors (SRFB) suggests that females are more sensitive to facial expressions than are males, and should detect facial expressions more quickly. A re-examination of recent research with children demonstrates that girls do detect various facial expressions more quickly than do boys. Although this provides support for SRFB, further examination of SRFB in children would lend important support this evolutionary-based theory.

In this article we discuss the aspects of designing facial expressions for virtual humans (VHs) with a specific culture. First we explore the notion of cultures and its relevance for applications with a VH. Then we give a general scheme of designing emotional facial expressions, and identify the stages where a human is involved, either as a real person with some specific role, or as a VH displaying facial expressions. We discuss how the (...) display and the emotional meaning of facial expressions may be measured in objective ways, and how the culture of displayers and the judges may influence the process of analyzing human facial expressions and evaluating synthesized ones. We review psychological experiments on cross-cultural perception of emotional facial expressions. By identifying the culturally critical issues of data collection and interpretation with both real and VHs, we aim at providing a methodological reference and inspiration for further research. (shrink)

ABSTRACTWe investigated whether moral violations involving harm selectively elicit anger, whereas purity violations selectively elicit disgust, as predicted by the Moral Foundations Theory. We analysed participants’ spontaneous facial expressions as they listened to scenarios depicting moral violations of harm and purity. As predicted by MFT, anger reactions were elicited more frequently by harmful than by impure actions. However, violations of purity elicited more smiling reactions and expressions of anger than of disgust. This effect was found both in (...) a classic set of scenarios and in a new set in which the different kinds of violations were matched on weirdness. Overall, these findings are at odds with predictions derived from MFT and provide support for “monist” accounts that posit harm at the basis of all moral violations. However, we found that smiles were differentially linked to purity violations, which leaves open the possibility of distinct moral modules. (shrink)

Following Yik and Russell (1999) a judgement paradigm was used to examine to what extent differential accuracy of recognition of facial expressions allows evaluation of the well-foundedness of different theoretical views on emotional expression. Observers judged photos showing facial expressions of seven emotions on the basis of: (1) discrete emotion categories; (2) social message types; (3) appraisal results; or (4) action tendencies, and rated their confidence in making choices. Emotion categories and appraisals were judged significantly (...) more accurately and confidently than messages or action tendencies. These results do not support claims of primacy for message or action tendency views of facial expression. Based on a componential model of emotion it is suggested that judges can infer components from categories and vice versa. (shrink)

Methodological problems often arise when a special case is confused with the general principle. So you will find affordances only for ‚artifacts’ if you restrict the analysis to ‚artifacts’. The general principle, however, is an ‚invitation character’, which triggers an action. Consequently, an action-theoretical approach known as ‚pragmatic turn’ in cognitive science is recommended. According to this approach, the human being is not a passive-receptive being but actively produces those action effects that open up the world to us. This ‚ideomotor (...) approach’ focuses on the so-called ‚epistemic actions’, which guide our perception as conscious and unconscious cognitions. Due to ‚embodied cognition’ the own body is assigned an indispensable role. The action theoretical approach of ‚enactive cognition’ enables that every form can be consistently processualized. Thus, each ‚Gestalt’ is understood as the process result of interlocking cognitions of ‚forward modelling’ and ‚inverse modelling’. As can be shown, these cognitions are fed by previous experiences of real interaction, which later changes into a mental trial treatment, which is highly automated and can therefore take place unconsciously. It is now central that every object may have such affordances that call for instrumental or epistemic action. In the simplest case, it is the body and the facial expressions of our counterpart that can be understood as a question and provoke an answer/reaction. Thus, emotion is not only to be understood as expression/output according to the scheme ‚input-processing-output’, but acts itself as a provocative act/input. Consequently, artifacts are neither necessary nor sufficient conditions for affordances. Rather, they exist in all areas of cognition—from Enactive Cognition to Social Cognition. (shrink)

Gaze plays a fundamental role in the processing of facial expressions from birth. Gaze direction is a crucial part of the social signal encoded in and decoded from faces. The ability to discriminate gaze direction, already evident early in life, is essential for the development of more complex socially relevant tasks, such as joint and shared attention. At the same time, facial expressions play a fundamental role in the encoding of gaze direction and, when combined, expression (...) and gaze communicate behavioural motivation to approach or avoid. However, the investigation of how gaze direction and emotional expression interact during the processing of a face has been relatively neglected, and is the key question of this review. (shrink)

In honoring Carroll Izard’s contributions to emotion research, we discuss infant facial activity and emotion expression. We consider the debated issue of whether infants are biologically prepared to express specific emotions. We offer a perspective that potentially integrates differing viewpoints on infant facial expression of emotion. Specifically, we suggest that evolution has prepared infants with innate action readiness patterns, which are crucial for early infant–caregiver social interaction, and in the course of social interaction specific facial configurations acquire (...) functional significance, becoming associated with specific emotions. Research has not confirmed the presence of innate neurophysiological action patterns that map onto discrete emotions but evidence indicates that the possibility has not been ruled out. (shrink)

A substantial body of research has established that even when we are not consciously aware of the faces of others we are nevertheless sensitive to, and impacted by their facial expression. In this paper, we consider this body of research from a new perspective by examining the functions of unconscious perception revealed by these studies. A consideration of the literature from this perspective highlights that existing research methods are limited when it comes to revealing possible functions of unconscious perception. (...) The critical shortcoming is that in all of the methods, the perceived facial expression remains outside of awareness. This is a problem because there are good reasons to believe that one important function of unconsciously perceived negative faces is to attract attention so that they are consciously perceived; such conscious perception, however, is never allowed with existing methodologies. We discuss recent studies of emotional face perception under conditions of visual search that address this issue directly. Further, we suggest that methodologies that do not examine cognitive processes as they occur in more natural settings may result in fundamental misunderstandings of human cognition. (shrink)

In the target article, we reviewed empirical evidence regarding the relationship between facial expressions and emotion in infancy. In our response to commentators, we make three main points. First, we concur with Hertenstein that the field has thus far relied too heavily on deductive reasoning, and suggest that future research strike a balance between inductive and deductive reasoning. Second, we maintain that infant recognition of discrete emotions remains an open question. Third, we state our position regarding the revised (...) version of DET. (shrink)

Facial expressions of emotion play an important role in human social interactions. However, posed expressions of emotion are not always the same as genuine feelings. Recent research has found that facial expressions are increasingly used as a tool for understanding social interactions instead of personal emotions. Therefore, the credibility assessment of facial expressions, namely, the discrimination of genuine (spontaneous) expressions from posed (deliberate/volitional/deceptive) ones, is a crucial yet challenging task in facial (...) expression understanding. With recent advances in computer vision and machine learning techniques, rapid progress has been made in recent years for automatic detection of genuine and posed facial expressions. This paper presents a general review of the relevant research, including several spontaneous vs. posed (SVP) facial expression databases and various computer vision based detection methods. In addition, a variety of factors that will influence the performance of SVP detection methods are discussed along with open issues and technical challenges in this nascent field. (shrink)

Evidence on the coherence between emotion and facial expression in adults from laboratory experiments is reviewed. High coherence has been found in several studies between amusement and smiling; low to moderate coherence between other positive emotions and smiling. The available evidence for surprise and disgust suggests that these emotions are accompanied by their “traditional” facial expressions, and even components of these expressions, only in a minority of cases. Evidence concerning sadness, anger, and fear is very limited. (...) For sadness, one study suggests that high emotion–expression coherence may exist in specific situations, whereas for anger and fear, the evidence points to low coherence. Insufficient emotion intensity and inhibition of facial expressions seem unable to account for the observed dissociations between emotion and facial expression. (shrink)

With a few yet increasing number of exceptions, the cognitive sciences enthusiastically endorsed the idea that there are basic facial expressions of emotions that are created by specific configurations of facial muscles. We review evidence that suggests an inherent role for context in emotion perception. Context does not merely change emotion perception at the edges; it leads to radical categorical changes. The reviewed findings suggest that configurations of facial muscles are inherently ambiguous, and they call for (...) a different approach towards the understanding of facial expressions of emotions. Prices of sticking with the modal view, and advantages of an expanded view, are succinctly reviewed. (shrink)

Although cognitive science was multidisciplinary from the start, an under-emphasis on anthropology has left the field with limited research in small scale, indigenous societies. Neglecting the anthropological perspective is risky, given that once-canonical cognitive science findings have often been shown to be artifacts of enculturation rather than cognitive universals. This imbalance has become more problematic as the increased use of Western theory-driven approaches, many of which assume human uniformity (“universality”), confronts the absence of a robust descriptive base that might provide (...) clarifying or even contrary evidence. We highlight the need for remedies to such shortcomings by suggesting a two-fold methodological shift. First, studies conducted in indigenous societies can benefit by relying on multidisciplinary research groups to diminish ethnocentrism and enhance the quality of the data. Second, studies devised for Western societies can readily be adapted to the changing settings encountered in the field. Here, we provide examples, drawn from the areas of emotion and facial expressions, to illustrate potential solutions to recurrent problems in enhancing the quality of data collection, hypothesis testing, and the interpretation of results. (shrink)