Facial expression recognition plays a significant part in artificial intelligence and computer vision. However, most of facial expression recognition methods have not obtained satisfactory results based on low-level features. The existed methods used in facial expression recognition encountered the major issues of linear inseparability, large computational burden, and data redundancy. To obtain satisfactory results, we propose an innovative deep learning model using the kernel entropy component analysis network and directed acyclic graph support vector machine. We use (...) the KECANet in the feature extraction stage. In the stage of output, binary hashing and blockwise histograms are adopted. We sent the final output features to the DAGSVM classifier for expression recognition. We test the performance of our proposed method on three databases of CK+, JAFFE, and CMU Multi-PIE. According to the experiment results, the proposed method can learn high-level features and provide more recognition information in the stage of training, obtaining a higher recognition rate. (shrink)

The current meta-analysis reviews 24 studies on self-reported emotional reactions to facial expressions in socially anxious versus nonanxious individuals. We hypothesized that socially anxious individuals would perceive all face types as less approachable, more negative, and more arousing. After correcting for biases, results showed that socially anxious individuals, compared to controls, reported lower approachability to all types of expressions and higher arousal in response to neutral expressions. Variances among effects usually could not be explained by (...) the proposed moderators. This suggests that current conceptualizations of social anxiety should take into account the willingness to approach social stimuli rather than global measures of emotion or arousal. (shrink)

Facial expression is a mode of close‐proximity non‐vocal communication used by primates and is produced by mimetic/facial musculature. Arguably, primates make the most‐intricate facial displays and have some of the most‐complex facial musculature of all mammals. Most of the earlier ideas of primate mimetic musculature, involving its function in facial displays and its evolution, were essentially linear “scala natural” models of increasing complexity. More‐recent work has challenged these ideas, suggesting that ecological factors and social systems (...) have played a much larger role in explaining the diversity of structures than previously believed. The present review synthesizes the evidence from gross muscular, microanatomical, behavioral and neurobiological studies in order to provide a preliminary analysis of the factors responsible for the evolution of primate facial musculature with comparisons to general mammals. In addition, the unique structure, function and evolution of human mimetic musculature are discussed, along with the potential influential roles of human speech and eye gaze. BioEssays 30:212–225, 2008. © 2008 Wiley Periodicals, Inc. (shrink)

Automatic facial expression recognition has now advanced to the point that we are able to apply it to spontaneous expressions. The automated tools enable new research activity in cognitive neuroscience, psychiatry, education, human–machine communication, and human social dynamics. Moreover, automated facial expression analysis enables investigations into facial expression dynamics that were previously intractable by human coding because of the time required to code intensity changes. This article provides an overview on the state of the art (...) in computer vision approaches to facial expression recognition, including methods for characterizing expression dynamics. It then discusses behavioral studies that have employed automatic facial expression recognition to learn new information about the relationships of facial expression to internal state, and reviews the first generation of applications in learning and education that take advantage of the real-time expression signal. (shrink)

Facial expressions are a vital way for humans to show their perceived emotions. It is convenient for detecting and recognizing expressions or micro-expressions by annotating a lot of data in deep learning. However, the study of video-based expressions or micro-expressions requires that coders have professional knowledge and be familiar with action unit coding, leading to considerable difficulties. This paper aims to alleviate this situation. We deconstruct facial muscle movements from the motor cortex and (...) systematically sort out the relationship among facial muscles, AU, and emotion to make more people understand coding from the basic principles: We derived the relationship between AU and emotion based on a data-driven analysis of 5,000 images from the RAF-AU database, along with the experience of professional coders.We discussed the complex facial motor cortical network system that generates facial movement properties, detailing the facial nucleus and the motor system associated with facial expressions.The supporting physiological theory for AU labeling of emotions is obtained by adding facial muscle movements patterns.We present the detailed process of emotion labeling and the detection and recognition of AU.Based on the above research, the video's coding of spontaneous expressions and micro-expressions is concluded and prospected. (shrink)

Over the last few years, automatic facial micro-expression analysis has garnered increasing attention from experts across different disciplines because of its potential applications in various fields such as clinical diagnosis, forensic investigation and security systems. Advances in computer algorithms and video acquisition technology have rendered machine analysis of facial micro-expressions possible today. Although the study of facial micro-expressions is a well-established field in psychology, it is still relatively new from the computational perspective with (...) many interesting problems. In this survey, we present a comprehensive review of state-of-the-art databases and methods for micro-expressions spotting and recognition. Individual stages involved in the automation of these tasks are also described and reviewed at length. In addition, we also deliberate on the challenges and future directions in this growing field of automatic facial micro-expression analysis. (shrink)

This article discusses how research on the image-based analysis of facial images has informed this debate by demonstrating that a single representational system for facial identity and facial expression is not only computationally viable, but can simulate existing cognitive data demonstrating apparent dissociable processing of these two facial properties. It discusses the increasing number of cognitive studies that provide support for this view. Neuropsychological case studies of brain-injured patients and provide limited evidence for separate visual (...) routes processing facial identity and facial expression. It deals with neuroimaging studies addressing both facial identity and facial expression processing which shows that the fusiform gyrus contributes to the recognition of both facial properties. Research continues to reinforce the proposal that a common visual framework supports the representation of at least some aspects of facial identity and facial expression. (shrink)

We used multiple methods to examine two questions about emotion and culture: (1) Which facial expressions are recognised cross-culturally; and (2) does the “forced-choice” method lead to spurious findings of universality? Forty participants in the US and 40 in India were shown 14 facial expressions and asked to say what had happened to cause the person to make the face. Analyses of the social situations given and of the affect words spontaneously used showed high levels of (...) recognition for most of the expressions. A subsequent forced-choice task using the same faces confirmed these findings. Analysis of the pattern of magnitude, discreteness, and similarity of responses across cultures and expressions led to the conclusion that there is no neat distinction between cross-culturally recognisable and nonrecognisable expressions. Results are better described as a gradient of recognition. (shrink)

A substantial body of research has established that even when we are not consciously aware of the faces of others we are nevertheless sensitive to, and impacted by their facial expression. In this paper, we consider this body of research from a new perspective by examining the functions of unconscious perception revealed by these studies. A consideration of the literature from this perspective highlights that existing research methods are limited when it comes to revealing possible functions of unconscious perception. (...) The critical shortcoming is that in all of the methods, the perceived facial expression remains outside of awareness. This is a problem because there are good reasons to believe that one important function of unconsciously perceived negative faces is to attract attention so that they are consciously perceived; such conscious perception, however, is never allowed with existing methodologies. We discuss recent studies of emotional face perception under conditions of visual search that address this issue directly. Further, we suggest that methodologies that do not examine cognitive processes as they occur in more natural settings may result in fundamental misunderstandings of human cognition. (shrink)

We used multiple methods to examine two questions about emotion and culture: (1) Which facial expressions are recognised cross-culturally; and (2) does the “forced-choice” method lead to spurious findings of universality? Forty participants in the US and 40 in India were shown 14 facial expressions and asked to say what had happened to cause the person to make the face. Analyses of the social situations given and of the affect words spontaneously used showed high levels of (...) recognition for most of the expressions. A subsequent forced-choice task using the same faces confirmed these findings. Analysis of the pattern of magnitude, discreteness, and similarity of responses across cultures and expressions led to the conclusion that there is no neat distinction between cross-culturally recognisable and nonrecognisable expressions. Results are better described as a gradient of recognition. (shrink)

This paper points out that a major shift of paradigm is currently going on in the study of the human face and it seeks to articulate and to develop the fundamental assumptions underlying this shift. The main theses of the paper are: 1) Facial expressions can convey meanings comparable to the meanings of verbal utterances. 2) Semantic analysis (whether of verbal utterances or of facial expressions) must distinguish between the context-independent invariant and its contextual interpretations. (...) 3) Certain components of facial behavior (¿facial gestures¿) do have constant context-independent meanings. 4) The meanings of facial components and configurations of components have an inherent first-person and present tense orientation. 5) The basis for the interpretation of facial gestures is, above all, experiential. 6) The meanings of some facial expressions are universally intelligible and can be interpreted without reference to any local conventions. 7) To be fruitful, the semantic analysis of facial expressions needs a methodology. This can be derived from the methodological experience of linguistic semantics. The author illustrates and supports these theses by analyzing a range of universally interpretable facial expressions such as the following ones: ¿brow furrowed¿ (i.e. eyebrows drawn together); eyebrows raised; eyes wide open; corners of the mouth raised; corners of the mouth lowered; mouth open (while not speaking); lips pressed together; upper lip and nose ¿raised¿ (and, consequently, nose wrinkled). (shrink)

This paper points out that a major shift of paradigm is currently going on in the study of the human face and it seeks to articulate and to develop the fundamental assumptions underlying this shift. The main theses of the paper are: 1) Facial expressions can convey meanings comparable to the meanings of verbal utterances. 2) Semantic analysis must distinguish between the context-independent invariant and its contextual interpretations. 3) Certain components of facial behavior do have constant (...) context-independent meanings. 4) The meanings of facial components and configurations of components have an inherent first-person and present tense orientation. 5) The basis for the interpretation of facial gestures is, above all, experiential. 6) The meanings of some facial expressions are universally intelligible and can be interpreted without reference to any local conventions. 7) To be fruitful, the semantic analysis of facial expressions needs a methodology. This can be derived from the methodological experience of linguistic semantics. The author illustrates and supports these theses by analyzing a range of universally interpretable facial expressions such as the following ones: “brow furrowed” ; eyebrows raised; eyes wide open; corners of the mouth raised; corners of the mouth lowered; mouth open ; lips pressed together; upper lip and nose “raised”. (shrink)

Facial expressions of emotion play an important role in human social interactions. However, posed expressions of emotion are not always the same as genuine feelings. Recent research has found that facial expressions are increasingly used as a tool for understanding social interactions instead of personal emotions. Therefore, the credibility assessment of facial expressions, namely, the discrimination of genuine (spontaneous) expressions from posed (deliberate/volitional/deceptive) ones, is a crucial yet challenging task in facial (...) expression understanding. With recent advances in computer vision and machine learning techniques, rapid progress has been made in recent years for automatic detection of genuine and posed facial expressions. This paper presents a general review of the relevant research, including several spontaneous vs. posed (SVP) facial expression databases and various computer vision based detection methods. In addition, a variety of factors that will influence the performance of SVP detection methods are discussed along with open issues and technical challenges in this nascent field. (shrink)

Recent work in social robotics, which is aimed both at creating an artificial intelligence and providing a test-bed for psychological theories of human social development, involves building robots that can learn from ‘face-to-face’ interaction with human beings — as human infants do. The building-blocks of this interaction include the robot’s ‘expressive’ behaviours, for example, facial-expression and head-and-neck gesture. There is here an ideal opportunity to apply Wittgensteinian conceptual analysis to current theoretical and empirical work in the sciences. Wittgenstein’s (...) philosophical psychology is sympathetic to embodied and situated Artificial Intelligence (see Proudfoot, 2002, 2004b), and his discussion of facial-expression is remarkably modern. In this chapter, I explore his approach to facial-expression, using smiling as a representative example, and apply it to the canonical interactive face robot, Cynthia Breazeal’s Kismet (see e.g. Breazeal, 2009, 2002). I assess the claim that Kismet has expressive behaviours, with the aim of generating philosophical insights for AI. (shrink)

Background: This study developed a photo and video database of 4-to-6-year-olds expressing the seven induced and posed universal emotions and a neutral expression. Children participated in photo and video sessions designed to elicit the emotions, and the resulting images were further assessed by independent judges in two rounds. Methods: In the first round, two independent judges, experts in the Facial Action Coding System, firstly analysed 3,668 emotions facial expressions stimuli from 132 children. Both judges reached 100% agreement (...) regarding 1,985 stimuli, which were then selected for a second round of analysis between judges 3 and 4. Results: The result was 1,985 stimuli were produced from 124 participants. A Kappa index of 0.70 and an accuracy of 73% between experts were observed. Lower accuracy was found for emotional expression by 4-year-olds than 6-year-olds. Happiness, disgust and contempt had the highest agreement. After a sub-analysis evaluation of all four judges, 100% agreement was reached for 1,381 stimuli which compound the ChildEFES database with 124 participants and 51% induced photographs. The number of stimuli of each emotion were: 87 for neutrality, 363 for happiness, 170 for disgust, 104 for surprise, 152 for fear, 144 for sadness, 157 for anger 157, and 183 for contempt. Conclusions: The findings show that this photo and video database can facilitate research on the mechanisms involved in early childhood recognition of facial emotions in children, contributing to the understanding of facial emotion recognition deficits which characterise several neurodevelopmental and psychiatric disorders. (shrink)

ERPs (event-related potentials) correlates are largely used in cognitive psychology and specifically for analysis of semantic information processing. Previous research has underlined a strong correlation between a negative-ongoing wave (N400), more frontally distributed, and semantic linguistic or extra-linguistic anomalies. With reference to the extra-linguistic domain, our experiment analyzed ERP variation in a semantic task of comprehension of emotional facial expressions. The experiment explored the effect of expectancy violation when subjects observed congruous or incongruous emotional facial patterns. (...) Four prototypical (anger, sadness, happiness and surprise) and four morphed faces were presented. Moreover, two distinct cognitive tasks (an implicit vs an explicit elaboration) were analyzed in order to evaluate the influence of spontaneous decoding in N400-like effects. An automatic, high-order cognitive process was found, elicited by a negative ERP variation similar to the linguistic N400 effect, which allows us to explain the congruous/incongruous decoding in semantic extra-linguistic comprehension. (shrink)

This article presents the analysis of the main Spanish political candidates for the elections to be held on April 2019. The analysis focuses on the Facial Expression Analysis, a technique widely used in neuromarketing research. It allows to identify the micro-expressions that are very brief, involuntary. They are signals of hidden emotions that cannot be controlled voluntarily. The video with the final interventions of every candidate has been post-processed using the classification algorithms given by the (...) iMotions's AFFDEX platform. We have then analyzed these data. Firstly, we have identified and compare the basic emotions showed by each politician. Second, we have associated the basic emotions with specific moments of the candidate's speech, identifying the topics they address and relating them directly to the expressed emotion. Third, we have analyzed whether the differences shown by each candidate in every emotion are statistically significant. In this sense, we have applied the non-parametric chi-squared goodness-of-fit test. We have also considered the ANOVA analysis in order to test whether, on average, there are differences between the candidates. Finally, we have checked if there is consistency between the results provided by different surveys from the main media in Spain regarding the evaluation of the debate and those obtained in our empirical analysis. A predominance of negative emotions has been observed. Some inconsistencies were found between the emotion expressed in the facial expression and the verbal content of the message. Also, evidences got from statistical analysis confirm that the differences observed between the various candidates with respect to the basic emotions, on average, are statistically significant. In this sense, this article provides a methodological contribution to the analysis of the public figures' communication, which could help politicians to improve the effectiveness of their messages identifying and evaluating the intensity of the expressed emotions. (shrink)

Facial expressions are one of the commonly used implicit measurements for the in-vehicle affective computing. However, the time courses and the underlying mechanism of facial expressions so far have been barely focused on. According to the Component Process Model of emotions, facial expressions are the result of an individual's appraisals, which are supposed to happen in sequence. Therefore, a multidimensional and dynamic analysis of drivers' fear by using facial expression data could profit (...) from a consideration of these appraisals. A driving simulator experiment with 37 participants was conducted, in which fear and relaxation were induced. It was found that the facial expression indicators of high novelty and low power appraisals were significantly activated after a fear event. Furthermore, after the fear event, the activation of high novelty occurred earlier than low power. These results suggest that multidimensional analysis of facial expression is suitable as an approach for the in-vehicle measurement of the drivers' emotions. Furthermore, a dynamic analysis of drivers' facial expressions considering of effects of appraisal components can add valuable information for the in-vehicle assessment of emotions. (shrink)

Methodological problems often arise when a special case is confused with the general principle. So you will find affordances only for ‚artifacts’ if you restrict the analysis to ‚artifacts’. The general principle, however, is an ‚invitation character’, which triggers an action. Consequently, an action-theoretical approach known as ‚pragmatic turn’ in cognitive science is recommended. According to this approach, the human being is not a passive-receptive being but actively produces those action effects that open up the world to us. This (...) ‚ideomotor approach’ focuses on the so-called ‚epistemic actions’, which guide our perception as conscious and unconscious cognitions. Due to ‚embodied cognition’ the own body is assigned an indispensable role. The action theoretical approach of ‚enactive cognition’ enables that every form can be consistently processualized. Thus, each ‚Gestalt’ is understood as the process result of interlocking cognitions of ‚forward modelling’ and ‚inverse modelling’. As can be shown, these cognitions are fed by previous experiences of real interaction, which later changes into a mental trial treatment, which is highly automated and can therefore take place unconsciously. It is now central that every object may have such affordances that call for instrumental or epistemic action. In the simplest case, it is the body and the facial expressions of our counterpart that can be understood as a question and provoke an answer/reaction. Thus, emotion is not only to be understood as expression/output according to the scheme ‚input-processing-output’, but acts itself as a provocative act/input. Consequently, artifacts are neither necessary nor sufficient conditions for affordances. Rather, they exist in all areas of cognition—from Enactive Cognition to Social Cognition. (shrink)

State-of-the-art facial expression methods outperform human beings, especially, thanks to the success of convolutional neural networks. However, most of the existing works focus mainly on analyzing an adult’s face and ignore the important problems: how can we recognize facial expression from a baby’s face image and how difficult is it? In this paper, we first introduce a new face image database, named BabyExp, which contains 12,000 images from babies younger than two years old, and each image is with (...) one of three facial expressions. To the best of our knowledge, the proposed dataset is the first baby face dataset for analyzing a baby’s face image, which is complementary to the existing adult face datasets and can shed some light on exploring baby face analysis. We also propose a feature guided CNN method with a new loss function, called distance loss, to optimize interclass distance. In order to facilitate further research, we provide the benchmark of expression recognition on the BabyExp dataset. Experimental results show that the proposed network achieves the recognition accuracy of 87.90% on BabyExp. (shrink)

This article examines the significant relationship that existed between commercial British television and the study of animal behaviour. Ethological research provided important content for the new television channel, at the same time as that coverage played a substantial role in creating a new research specialism, the study of primate facial expressions, for this emergent scientific discipline. The key site in this was a television and film unit at London Zoo administered by the Zoological Society and Granada TV. The (...) Granada unit produced ‘Animal expressions’, a twenty-five-minute television film based on research on monkeys and apes by the Dutch postgraduate student and soon-to-be-leading-authority Jan van Hooff. Recovering the production and multiple uses of ‘Animal expressions’, this paper offers the first sustained historical analysis of science on commercial television. I show how Granada patronage helped Van Hooff to support his argument that human expressions such as smiling and laughter shared common evolutionary origins with similar facial movements in nonhuman primates. Emphasizing the mutual shaping of science and ITV, I argue that ‘Animal expressions’ repurposed televisual conventions of framing talking heads, and blended serious science with the comedy of ‘funny faces’, thereby epitomizing Granada's public-service strategy at a time when commercial television was defending itself from criticism in the Pilkington report. (shrink)

Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression is to characterize the visual features of specific facial expressions in individual cultures. Here we developed an image analysis framework for this purpose using convolutional neural networks that through training learned visual features critical for classification. We analyzed photographs of facial (...) class='Hi'>expressions derived from two databases, each developed in a different country, in which corresponding emotion labels were available. While the CNNs reached high rates of correct results that were far above chance after training with each database, they showed many misclassifications when they analyzed faces from the database that was not used for training. These results suggest that facial features useful for classifying facial expressions differed between the databases. The selectivity of computational units in the CNNs to action units of the face varied across the facial expressions. Importantly, the AU selectivity often differed drastically between the CNNs trained with the different databases. Similarity and dissimilarity of these tuning profiles partly explained the pattern of misclassifications, suggesting that the AUs are important for characterizing the facial features and differ between the two countries. The AU tuning profiles, especially those reduced by principal component analysis, are compact summaries useful for comparisons across different databases, and thus might advance our understanding of universality vs. specificity of facial expressions across cultures. (shrink)

An assessment of mood or emotion is important in developing mental health measures, and facial expressions are strongly related to mood or emotion. This study thus aimed to examine the relationship between levels of negative mood and characteristics of mouth parts when moods are drawn as facial expressions on a common platform. A cross-sectional study of Japanese college freshmen was conducted, and 1,068 valid responses were analyzed. The questionnaire survey consisted of participants’ characteristics, the Profile of (...) Mood States, and a sheet of facial expression drawing, and the sheet was digitized and analyzed using an image-analysis software. Based on the total POMS score as an index of negative mood, the participants were divided into four groups: low, normal, high, and very high. Lengths of drawn lines and between both mouth corners were significantly longer, and circularity and roundness were significantly higher in the L group. With increasing levels of negative mood, significant decreasing trends were observed in these lengths. Convex downward and enclosed figures were significantly predominant in the L group, while convex upward figures were significantly predominant and a tendency toward predominance of no drawn mouths or line figures was found in the H and VH groups. Our results suggest that mood states can be significantly related to the size and figure characteristics of drawn mouths of FACED on a non-verbal common platform. That is, these findings mean that subjects with low negative mood may draw a greater and rounder mouth and figures that may be enclosed and downward convex, while subjects with a high negative mood may not draw the line, or if any, may draw the line shorter and upward convex. (shrink)

This paper presents an analytical framework for analyzing how multimodal resources of emotion expression are semiotically materialized in discursive interactions specific to political discourse. Interested in how political personae are emotionally constructed through multimodal meaning-making practices, our analysis model assumes an interdisciplinary perspective, which integrates facial expression analysis – using FaceReader™ software –, the theory of emotional arcs and bodily actions (hand gestures) analysis that express emotions, in the analytical framework of multimodality. The results show how (...) the multimodal choices that political actors make during discursive interactions allow them to build their political brand and make connections with the audience on an emotional level. (shrink)

To characterize human emotions, researchers have increasingly utilized Automatic Facial Expression Analysis (AFEA), which automates the Facial Action Coding System (FACS) and translates the facial muscular positioning into the basic universal emotions. There is broad interest in the application of FACS for assessing consumer expressions as an indication of emotions to consumer product-stimuli. However, the translation of FACS to characterization of emotions is elusive in the literature. The aim of this systematic review is to give (...) an overview of how FACS has been used to investigate human emotional behavior to consumer product-based stimuli. The search was limited to studies published in English after 1978, conducted on humans, using FACS or its action units to investigate affect, where emotional response is elicited by consumer product-based stimuli evoking at least one of the five senses. The search resulted in an initial total of 1,935 records, of which 55 studies were extracted and categorized based on the outcomes of interest including (i) method of FACS implementation; (ii) purpose of study; (iii) consumer product-based stimuli used; and (iv) measures of affect validation. Most studies implemented FACS manually (73%) to develop products and/or software (20%) and used consumer product-based stimuli that had known and/or defined capacity to evoke a particular affective response, such as films and/or movie clips (20%); minimal attention was paid to consumer products with low levels of emotional competence or with unknown affective impact. The vast majority of studies (53%) did not validate FACS-determined affect and, of the validation measures that were used, most tended to be discontinuous in nature and only captured affect as it holistically related to an experience. This review illuminated some inconsistencies in how FACS is carried out as well as how emotional response is inferred from facial muscle activation. This may prompt researchers to consider measuring the total consumer experience by employing a variety of methodologies in addition to FACS and its emotion-based interpretation guide. Such strategies may better conceptualize consumers' experience with products of low, unknown, and/or undefined capacity to evoke an affective response such as product prototypes, line extensions, etc. (shrink)

An analysis of the identity issues involved in facial allograft transplantation is provided in this paper. The identity issues involved in organ transplantation in general, under both theoretical accounts of personal identity and subjective accounts provided by organ recipients, are examined. It is argued that the identity issues involved in facial allograft transplantation are similar to those involved in organ transplantation in general, but much stronger because the face is so closely linked with personal identity. Recipients of (...) facial allograft transplantation have the potential to feel that their identity is a mix between their own and the donor’s, and the donor’s family is potentially likely to feel that their loved one “lives on”. It is also argued that facial allograft transplantation allows the recipients to regain an identity, because they can now be seen in the social world. Moreover, they may regain expressivity, allowing for them to be seen even more by others, and to regain an identity to an even greater extent. Informing both recipients and donors about the role that identity plays in facial allograft transplantation could enhance the consent process for facial allograft transplantation and donation. (shrink)

An analysis of the identity issues involved in facial allograft transplantation is provided in this paper. The identity issues involved in organ transplantation in general, under both theoretical accounts of personal identity and subjective accounts provided by organ recipients, are examined. It is argued that the identity issues involved in facial allograft transplantation are similar to those involved in organ transplantation in general, but much stronger because the face is so closely linked with personal identity. Recipients of (...) facial allograft transplantation have the potential to feel that their identity is a mix between their own and the donor’s, and the donor’s family is potentially likely to feel that their loved one ‘‘lives on’’. It is also argued that facial allograft transplantation allows the recipients to regain an identity, because they can now be seen in the social world. Moreover, they may regain expressivity, allowing for them to be seen even more by others, and to regain an identity to an even greater extent. Informing both recipients and donors about the role that identity plays in facial allograft transplantation could enhance the consent process for facial allograft transplantation and donation. (shrink)

Most research on cross-cultural emotion recognition has focused on facial expressions. To integrate the body of evidence on vocal expression, we present a meta-analysis of 37 cross-cultural studies...

In an attempt to discover the facial action units for affective states that occur during complex learning, this study adopted an emote-aloud procedure in which participants were recorded as they verbalised their affective states while interacting with an intelligent tutoring system (AutoTutor). Participants’ facial expressions were coded by two expert raters using Ekman's Facial Action Coding System and analysed using association rule mining techniques. The two expert raters received an overall kappa that ranged between.76 and.84. The (...) association rule mining analysis uncovered facial actions associated with confusion, frustration, and boredom. We discuss these rules and the prospects of enhancing AutoTutor with non-intrusive affect-sensitive capabilities. (shrink)

Signed languages employ finely articulated facial and head displays to express grammatical meanings such as mood and modality, complex propositions, information structure, assertions, content and yes/no questions, imperatives, and miratives. In this paper we examine two facial displays: an upper face display in which the eyebrows are pulled together called brow furrow, and a lower face display in which the corners of the mouth are turned down into a distinctive configuration that resembles a frown or upside-down U-shape. Our (...) analysis employs Cognitive Grammar, specifically the control cycle and its manifestation in effective control and epistemic control. Our claim is that effective and epistemic control are associated with embodied actions. Prototypical physical effective control requires effortful activity and the forceful exertion of energy and is commonly correlated with upper face activity, often called the “face of effort.” The lower face display has been shown to be associated with epistemic indetermination, uncertainty, doubt, obviousness, and skepticism. We demonstrate that the control cycle unifies the diverse grammatical functions expressed by each facial display within a language, and that they express similar functions across a wide range of signed languages. (shrink)

Psychiatric genomics research protocols are increasingly incorporating tools of deep phenotyping to observe and examine phenotypic abnormalities among individuals with neurodevelopmental disorders. In particular, photography and the use of two-dimensional and three-dimensional facial analysis is thought to shed further light on the phenotypic expression of the genes underlying neurodevelopmental disorders, as well as provide potential diagnostic tools for clinicians. In this paper, I argue that the research use of photography to aid facial phenotyping raises deeply fraught issues (...) from an ethical point of view. First, the process of objectification through photographic imagery and facial analysis could potentially worsen the stigmatisation of persons with neurodevelopmental disorders. Second, the use of photography for facial phenotyping has worrying parallels with the historical misuse of photography to advance positive and negative eugenics around race, ethnicity and intellectual disability. The paper recommends ethical caution in the use of photography and facial phenotyping in psychiatric genomics studies exploring neurodevelopmental disorders, outlining certain necessary safeguards, such as a critical awareness of the history of anthropometric photography use among scientists, as well as the exploration of photographic methodologies that could potentially empower individuals with disabilities. (shrink)

Resumen El reconocimiento facial es una tecnología que se ha utilizado para el control de la población a fin de combatir los efectos de la pandemia por COVID-19. Sin embargo, esta tecnología se utiliza hace bastante tiempo en muchos ámbitos del quehacer económico, social y político. Este trabajo analiza el reconocimiento fa cial entendida como la captura de micro expresiones faciales mediante la cual se puede medir y evaluar emociones con el fin de vigilar movimiento, predecir com portamientos y (...) anticipar intenciones de compra y de voto. Nuestro objetivo es revisar los enfoques teóricos que sustentan esta tecnología que ha devenido en una nueva oportunidad de negocio, para proponer después algunas reflexiones éticas derivadas.Facial recognition is a technology that is used to control people in order to fight the impacts of COVID-19. However, this technology has been used formany years in many economical, social and political areas. This work provides an analysis of the new business based on technologies of facial micro-expressions recognition. By means of this procedure, the involved companies measure and evaluate emotions with the aim of monitoring the movement, predicting behaviors, intentions of purchase and vote, between other applications. Our purpose is to check the theoretical suppositions that sustain this business, to propose later some ethical derivative reflections. (shrink)

Emotional facial expression are an important communication channel between artificial characters and their users. Humans are trained to perceive emotions. Robots and virtual agents can use them to make their inner states transparent. Literature reported that some emotional types, such as anger, are perceived as being more intense than others. Other studies indicated that gender influences the perception. Our study shows that once the individual differences amongst participants are included in the statistical analysis, then the emotion type has (...) no further explanatory power. Artificial characters therefore should adapt to their specific users. (shrink)

This paper presents a study of modality in Iranian Sign Language (ZEI) from a cognitive perspective, aimed at analyzing two linguistic channels: facial and manual. While facial markers and their grammatical functions have been studied in some sign languages, we have few detailed analyses of the facial channel in comparison with the manual channel in conveying modal concepts. This study focuses on the interaction between manual and facial markers. A description of manual modal signs is offered. (...) Three facial markers and their modality values are also examined: squinted eyes, brow furrow, and downward movement of lip corners (horseshoe mouth). In addition to offering this first descriptive analysis of modality in ZEI, this paper also applies the Cognitive Grammar model of modality, the Control Cycle, and the Reality Model, classifying modals into two kinds, effective and epistemic. It is suggested that effective control, including effective modality, tends to be expressed on the hands, while facial markers play an important role in marking epistemic assessment, one manifestation of which is epistemic modality. ZEI, like some other sign languages, exhibits an asymmetry between the number of manual signs and facial markers expressing epistemic modality: while the face can be active in the expression of effective modality, it is commonly the only means of expressing epistemic modality. By positing an epistemic core in effective modality, Cognitive Grammar provides a theoretical basis for these findings. (shrink)

This paper proposes that human expression of pain in the presence or absence of caregivers, and the detection of pain by observers, arises from evolved propensities. The function of pain is to demand attention and prioritise escape, recovery, and healing; where others can help achieve these goals, effective communication of pain is required. Evidence is reviewed of a distinct and specific facial expression of pain from infancy to old age, consistent across stimuli, and recognizable as pain by observers. Voluntary (...) control over amplitude is incomplete, and observers can better detect pain that the individual attempts to suppress rather than amplify or simulate. In many clinical and experimental settings, the facial expression of pain is incorporated with verbal and nonverbal vocal activity, posture, and movement in an overall category of pain behaviour. This is assumed by clinicians to be under operant control of social contingencies such as sympathy, caregiving, and practical help; thus, strong facial expression is presumed to constitute an attempt to manipulate these contingencies by amplification of the normal expression. Operant formulations support skepticism about the presence or extent of pain, judgments of malingering, and sometimes the withholding of caregiving and help. To the extent that pain expression is influenced by environmental contingencies, however, could equally plausibly constitute the release of suppression according to evolved contingent propensities that guide behaviour. Pain has been largely neglected in the evolutionary literature and the literature on expression of emotion, but an evolutionary account can generate improved assessment of pain and reactions to it. (shrink)

Research on facial emotion expression has mostly focused on emotion recognition, assuming that a small number of discrete emotions is elicited and expressed via prototypical facial muscle configurations as captured in still photographs. These are expected to be recognized by observers, presumably via template matching. In contrast, appraisal theories of emotion propose a more dynamic approach, suggesting that specific elements of facial expressions are directly produced by the result of certain appraisals and predicting the facial (...) patterns to be expected for certain appraisal configurations. This approach has recently been extended to emotion perception, claiming that observers first infer individual appraisals and only then make categorical emotion judgments based on the estimated appraisal patterns, using semantic inference rules. Here we report two related studies to empirically investigate the facial action unit configurations that are used by actors to convey specific emotions in short affect bursts and to examine to what extent observers can infer a person’s emotions from the predicted facial expression configurations. The results show that 1) professional actors use many of the predicted facial action unit patterns to enact systematically specified appraisal outcomes in a realistic scenario setting, and 2) naïve observers infer the respective emotions based on highly similar facial movement configurations with a degree of accuracy comparable to earlier research findings. Based on estimates of underlying appraisal criteria for the different emotions we conclude that the patterns of facial action units identified in this research correspond largely to prior predictions and encourage further research on appraisal-driven expression and inference. (shrink)

In this article, we review empirical evidence regarding the relationship between facial expression and emotion during infancy. We focus on differential emotions theory’s view of this relationship because of its theoretical and methodological prominence. We conclude that current evidence fails to support its proposal regarding a set of pre-specified facial expressions that invariably reflect a corresponding set of discrete emotions in infants. Instead, the relationship between facial expression and emotion appears to be more complex. Some (...) class='Hi'>facial expressions may have different meanings in infants than in children and adults. In addition, nonemotion factors may sometimes lead to the production of “emotional” facial expressions. We consider alternative perspectives on the nature of emotion and emotional expression in infancy with particular focus on differentiation and dynamical systems approaches. (shrink)

Following Yik and Russell (1999) a judgement paradigm was used to examine to what extent differential accuracy of recognition of facial expressions allows evaluation of the well-foundedness of different theoretical views on emotional expression. Observers judged photos showing facial expressions of seven emotions on the basis of: (1) discrete emotion categories; (2) social message types; (3) appraisal results; or (4) action tendencies, and rated their confidence in making choices. Emotion categories and appraisals were judged significantly more (...) accurately and confidently than messages or action tendencies. These results do not support claims of primacy for message or action tendency views of facial expression. Based on a componential model of emotion it is suggested that judges can infer components from categories and vice versa. (shrink)