There is debate within the literature as to whether emotion dysregulation (ED) in Attention-Deficit Hyperactivity Disorder (ADHD) reflects deviant attentional mechanisms or atypical perceptual emotion processing. Previous reviews have reliably examined the nature of facial, but not vocal, emotion recognition accuracy in ADHD. The present meta-analysis quantified vocal emotion recognition (VER) accuracy scores in ADHD and controls using robust variance estimation, gathered from 21 published and unpublished papers. Additional moderator analyses were carried (...) out to determine whether the nature of VER accuracy in ADHD varied depending on emotion type. Findings revealed a medium effect size for the presence of VER deficits in ADHD, and moderator analyses showed VER accuracy in ADHD did not differ due to emotion type. These results support the theories which implicate the role of attentional mechanisms in driving VER deficits in ADHD. However, there is insufficient data within the behavioural VER literature to support the presence of emotion processing atypicalities in ADHD. Future neuro-imaging research could explore the interaction between attention and emotion processing in ADHD, taking into consideration ADHD subtypes and comorbidities. (shrink)

BackgroundWe review original papers on ovarian-hormone status in two areas of emotional processing: facial emotion recognition and emotional memory. Ovarian-hormone status is operationalized by the levels of the steroid sex hormones 17β-estradiol and progesterone, fluctuating over the natural menstrual cycle and suppressed under oral contraceptive use. We extend previous reviews addressing single areas of emotional processing. Moreover, we systematically examine the role of stimulus features such as emotion type or stimulus valence and aim at elucidating factors that (...) reconcile the inconsistent results.MethodsWe followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and included papers published until September 2020 indexed in PubMed and Web of Science databases. Search terms were MeSH terms AND AND with representing our separately searched areas, resulting in, and. To be included, articles had to be written and published in English, examine healthy, non-pregnant adult women in their reproductive age, and measure or at least estimate levels of E2 and P4. In PubMed, the search was limited to humans and to the search term present in the title or abstract.ResultsFeatures of the provided stimulus material constitute a relevant influence that interacts with E2- and P4-related ovarian-hormone status. For instance, recognition of basic emotions appears to be more related to P4- than E2-levels. Quite consistent, OC intake was accompanied by impaired recognition accuracy of basic and also complex emotions, although not in a recent large-sample study assessing complex emotions. Memory recall of negative content was mainly enhanced by P4, especially after having been stressed.Discussion and ConclusionWe document the methodological diversity in the field, presumably contributing to the heterogeneity of results. More studies explicitly contrasting the early follicular phase, mid-cycle phase, mid-luteal, and OC intake while standardizing tasks are needed. Research would take advantage of using within-subject designs and accounting for the recognition of complex emotions. (shrink)

Linguistic communication is an important part of the cross-cultural perspective, and linguistic textual emotion recognition is a key massage in interpersonal communication. Spanish is the second largest language system in the world. The purpose of this paper is to identify the emotional features in Spanish texts. The improved BiLSTM framework is proposed. We select three widely used Spanish dictionaries as the datasets for our experiments, and then we finally obtain text sentiment classification results through text preprocessing, text (...) class='Hi'>emotion feature extraction, text topic detection, and emotion classification. We inserted the attention mechanism in the improved BiLSTM framework. It enables the shared feature encoder to obtain weighted representation results in the extraction of emotion features, which enhances the generalization ability of the model for text emotion feature recognition. Experimental results demonstrate that our approach performs better for specialized Spanish dictionary datasets. In terms of emotion recognition accuracy, the average value is as high as 76.21%. The overall performance outperforms current comparable machine learning methods and convolutional neural network methods. (shrink)

ABSTRACTRecently, Cognition and Emotion published an article demonstrating that age cues affect the speed and accuracy of emotion recognition. The authors claimed that the observed effect of target age on emotion recognition is better explained by evaluative than stereotype associations. Although we agree with their conclusion, we believe that with the research method the authors employed, it was impossible to detect a stereotype effect to begin with. In the current research, we successfully replicate previous (...) findings. Furthermore, by changing the comparative context, Study 2 provides a first test of age-stereotypes affecting emotions recognition. We discuss recommendations for future studies in the domain of social categorisation and emotion recognition. (shrink)

Emotion recognition plays an important part in human-computer interaction. Currently, the main challenge in electroencephalogram -based emotion recognition is the non-stationarity of EEG signals, which causes performance of the trained model decreasing over time. In this paper, we propose a two-level domain adaptation neural network to construct a transfer model for EEG-based emotion recognition. Specifically, deep features from the topological graph, which preserve topological information from EEG signals, are extracted using a deep neural network. (...) These features are then passed through TDANN for two-level domain confusion. The first level uses the maximum mean discrepancy to reduce the distribution discrepancy of deep features between source domain and target domain, and the second uses the domain adversarial neural network to force the deep features closer to their corresponding class centers. We evaluated the domain-transfer performance of the model on both our self-built data set and the public data set SEED. In the cross-day transfer experiment, the ability to accurately discriminate joy from other emotions was high: sadness, anger, and fear on the self-built data set. The accuracy reached 74.93% on the SEED data set. In the cross-subject transfer experiment, the ability to accurately discriminate joy from other emotions was equally high: sadness, anger, and fear on the self-built data set. The average accuracy reached 87.9% on the SEED data set, which was higher than WGAN-DA. The experimental results demonstrate that the proposed TDANN can effectively handle the domain transfer problem in EEG-based emotion recognition. (shrink)

In the human-computer interaction, electroencephalogram access for automatic emotion recognition is an effective way for robot brains to perceive human behavior. In order to improve the accuracy of the emotion recognition, a method of EEG access for emotion recognition based on a deep hybrid network was proposed in this paper. Firstly, the collected EEG was decomposed into four frequency band signals, and the multiscale sample entropy features of each frequency band were extracted. Secondly, (...) the constructed 3D MSE feature matrices were fed into a deep hybrid network for autonomous learning. The deep hybrid network was composed of a continuous convolutional neural network and hidden Markov models. Lastly, HMMs trained with multiple observation sequences were used to replace the artificial neural network classifier in the CNN, and the emotion recognition task was completed by HMM classifiers. The proposed method was applied to the DEAP dataset for emotion recognition experiments, and the average accuracy could achieve 79.77% on arousal, 83.09% on valence, and 81.83% on dominance. Compared with the latest related methods, the accuracy was improved by 0.99% on valence and 14.58% on dominance, which verified the effectiveness of the proposed method. (shrink)

Purpose This study responds to Agnieszka Landowska’s paper about the lack of accuracy in emotion recognition. Design/methodology/approach The approach is purely theoretical. The paper also refers to empirical studies. Findings The author first elaborates on Landowska’s “postulates” and then shortly expands on how virtual chatbots such as “AI therapists” pose considerable challenges to emotion recognition algorithms as well. Originality/value This viewpoint’s value is to elaborate and expand on an ongoing discussion on emotion recognition (...) technologies. (shrink)

This exploration aims to study the emotion recognition of speech and graphic visualization of expressions of learners under the intelligent learning environment of the Internet. After comparing the performance of several neural network algorithms related to deep learning, an improved convolution neural network-Bi-directional Long Short-Term Memory algorithm is proposed, and a simulation experiment is conducted to verify the performance of this algorithm. The experimental results indicate that the Accuracy of CNN-BiLSTM algorithm reported here reaches 98.75%, which is (...) at least 3.15% higher than that of other algorithms. Besides, the Recall is at least 7.13% higher than that of other algorithms, and the recognition rate is not less than 90%. Evidently, the improved CNN-BiLSTM algorithm can achieve good recognition results, and provide significant experimental reference for research on learners’ emotion recognition and graphic visualization of expressions in an intelligent learning environment. (shrink)

This study explored the priming effects of discriminated experiences on emotion recognition accuracy of Asian Americans. We hypothesized that when Asian Americans were reminded of discriminated experiences due to their race, they would detect subtle negative emotional expressions on White faces more accurately than would Asian Americans who were primed with a neutral topic. This priming effect was not expected to emerge in detecting negative facial expressions on Asian faces. To test this hypothesis, 108 participants were randomly (...) assigned to one of two conditions: write about their experiences with racial discrimination or write about a neutral topic. Then, they were given an emotion recognition test consisting of White and Asian faces. The current study found a significant interaction effect of priming condition by target race. When Asian Americans were reminded of discriminated experiences, they displayed heightened sensitivity to negative emotional expressions on White faces, but not to the negative expressions on Asian faces. The implications of these findings were discussed. (shrink)

Emotion recognition is an important part of building an intelligent human-computer interaction system and plays an important role in human-computer interaction. Often, people express their feelings through a variety of symbols, such as words and facial expressions. A business ecosystem is an economic community based on interacting organizations and individuals. Over time, they develop their capabilities and roles together and tend to develop themselves in the direction of one or more central enterprises. This paper aims to study a (...) multimodal ER method based on attention mechanism. It analyzes the current emotional state of consumers and the development direction of enterprises through multi-modal ER of human emotions and analysis of market trends, so as to provide the most appropriate response or plan. This paper firstly describes the related methods of multimodal ER and deep learning in detail, and briefly outlines the meaning of enterprise strategy in the business ecosystem. Then, two datasets, CMU-MOSI and CMU-MOSEI, are selected to design the scheme for multimodal ER based on self-attention mechanism. Through the comparative analysis of the accuracy of single-modal and multi-modal ER, the self-attention mechanism is applied in the experiment. The experimental results show that the average recognition accuracy of happy under multimodal ER reaches 91.5%. (shrink)

Emotion recognition, and in particular acial emotion recognition (FER), is among the most controversial applications of machine learning, not least because of its ethical implications for human subjects. In this article, we address the controversial conjecture that machines can read emotions from our facial expressions by asking whether this task can be performed reliably. This means, rather than considering the potential harms or scientific soundness of facial emotion recognition systems, focusing on the reliability of (...) the ground truths used to develop emotion recognition systems, assessing how well different human observers agree on the emotions they detect in subjects’ faces. Additionally, we discuss the extent to which sharing context can help observers agree on the emotions they perceive on subjects’ faces. Briefly, we demonstrate that when large and heterogeneous samples of observers are involved, the task of emotion detection from static images crumbles into inconsistency. We thus reveal that any endeavour to understand human behaviour from large sets of labelled patterns is over-ambitious, even if it were technically feasible. We conclude that we cannot speak of actual accuracy for facial emotion recognition systems for any practical purposes. (shrink)

Emotion recognition plays an important role in intelligent human–computer interaction, but the related research still faces the problems of low accuracy and subject dependence. In this paper, an open-source software toolbox called MindLink-Eumpy is developed to recognize emotions by integrating electroencephalogram and facial expression information. MindLink-Eumpy first applies a series of tools to automatically obtain physiological data from subjects and then analyzes the obtained facial expression data and EEG data, respectively, and finally fuses the two different signals (...) at a decision level. In the detection of facial expressions, the algorithm used by MindLink-Eumpy is a multitask convolutional neural network based on transfer learning technique. In the detection of EEG, MindLink-Eumpy provides two algorithms, including a subject-dependent model based on support vector machine and a subject-independent model based on long short-term memory network. In the decision-level fusion, weight enumerator and AdaBoost technique are applied to combine the predictions of SVM and CNN. We conducted two offline experiments on the Database for Emotion Analysis Using Physiological Signals dataset and the Multimodal Database for Affect Recognition and Implicit Tagging dataset, respectively, and conducted an online experiment on 15 healthy subjects. The results show that multimodal methods outperform single-modal methods in both offline and online experiments. In the subject-dependent condition, the multimodal method achieved an accuracy of 71.00% in the valence dimension and an accuracy of 72.14% in the arousal dimension. In the subject-independent condition, the LSTM-based method achieved an accuracy of 78.56% in the valence dimension and an accuracy of 77.22% in the arousal dimension. The feasibility and efficiency of MindLink-Eumpy for emotion recognition is thus demonstrated. (shrink)

Emotion recognition systems have been of interest to researchers for a long time. Improvement of brain-computer interface systems currently makes EEG-based emotion recognition more attractive. These systems try to develop strategies that are capable of recognizing emotions automatically. There are many approaches due to different features extractions methods for analyzing the EEG signals. Still, Since the brain is supposed to be a nonlinear dynamic system, it seems a nonlinear dynamic analysis tool may yield more convenient results. (...) A novel approach in Symbolic Time Series Analysis for signal phase space partitioning and symbol sequence generating is introduced in this study. Symbolic sequences have been produced by means of spherical partitioning of phase space; then, they have been compared and classified based on the maximum value of a similarity index. Obtaining the automatic independent emotion recognition EEG-based system has always been discussed because of the subject-dependent content of emotion. Here we introduce a subject-independent protocol to solve the generalization problem. To prove our method’s effectiveness, we used the DEAP dataset, and we reached an accuracy of 98.44% for classifying happiness from sadness. It was 93.75% for three, 89.06% for four, and 85% for five emotional groups. According to these results, it is evident that our subject-independent method is more accurate rather than many other methods in different studies. In addition, a subject-independent method has been proposed in this study, which is not considered in most of the studies in this field. (shrink)

Background: This study developed a photo and video database of 4-to-6-year-olds expressing the seven induced and posed universal emotions and a neutral expression. Children participated in photo and video sessions designed to elicit the emotions, and the resulting images were further assessed by independent judges in two rounds. Methods: In the first round, two independent judges, experts in the Facial Action Coding System, firstly analysed 3,668 emotions facial expressions stimuli from 132 children. Both judges reached 100% agreement regarding 1,985 stimuli, (...) which were then selected for a second round of analysis between judges 3 and 4. Results: The result was 1,985 stimuli were produced from 124 participants. A Kappa index of 0.70 and an accuracy of 73% between experts were observed. Lower accuracy was found for emotional expression by 4-year-olds than 6-year-olds. Happiness, disgust and contempt had the highest agreement. After a sub-analysis evaluation of all four judges, 100% agreement was reached for 1,381 stimuli which compound the ChildEFES database with 124 participants and 51% induced photographs. The number of stimuli of each emotion were: 87 for neutrality, 363 for happiness, 170 for disgust, 104 for surprise, 152 for fear, 144 for sadness, 157 for anger 157, and 183 for contempt. Conclusions: The findings show that this photo and video database can facilitate research on the mechanisms involved in early childhood recognition of facial emotions in children, contributing to the understanding of facial emotion recognition deficits which characterise several neurodevelopmental and psychiatric disorders. (shrink)

Due to the non-invasiveness and high precision of electroencephalography, the combination of EEG and artificial intelligence is often used for emotion recognition. However, the internal differences in EEG data have become an obstacle to classification accuracy. To solve this problem, considering labeled data from similar nature but different domains, domain adaptation usually provides an attractive option. Most of the existing researches aggregate the EEG data from different subjects and sessions as a source domain, which ignores the assumption (...) that the source has a certain marginal distribution. Moreover, existing methods often only align the representation distributions extracted from a single structure, and may only contain partial information. Therefore, we propose the multi-source and multi-representation adaptation for cross-domain EEG emotion recognition, which divides the EEG data from different subjects and sessions into multiple domains and aligns the distribution of multiple representations extracted from a hybrid structure. Two datasets, i.e., SEED and SEED IV, are used to validate the proposed method in cross-session and cross-subject transfer scenarios, experimental results demonstrate the superior performance of our model to state-of-the-art models in most settings. (shrink)

We focus on the role that motor mimicry plays in the SIMS model when interpreting whether a facial emotional expression is appropriate to an eliciting context. Based on our research, we find general support for the SIMS model in these situations, but with some qualifications on how disruption of motor mimicry as a process relates to speed and accuracy in judgments.

The conflicting findings from the few studies conducted with regard to gender differences in the recognition of vocal expressions of emotion have left the exact nature of these differences unclear. Several investigators have argued that a comprehensive understanding of gender differences in vocal emotion recognition can only be achieved by replicating these studies while accounting for influential factors such as stimulus type, gender-balanced samples, number of encoders, decoders and emotional categories. This study aimed to account for (...) these factors by investigating whether emotion recognition from vocal expressions differs as a function of both listeners’ and speakers’ gender. A total of N = 290 participants were randomly and equally allocated to two groups. One group listened to words and pseudo-words, while the other group listened to sentences and affect bursts. Participants were asked to categorize the stimuli with respect to the expressed emotions in a fixed-choice response format. Overall, females were more accurate than males when decoding vocal emotions, however, when testing for specific emotions these differences were small in magnitude. Speakers’ gender had a significant impact on how listeners’ judged emotions from the voice. The group listening to words and pseudo-words had higher identification rates for emotions spoken by male than by female actors, whereas in the group listening to sentences and affect bursts the identification rates were higher when emotions were uttered by female than male actors. The mixed pattern for emotion-specific effects, however, indicates that, in the vocal channel, the reliability of emotion judgments is not systematically influenced by speakers’ gender and the related stereotypes of emotional expressivity. Together, these results extend previous findings by showing effects of listeners’ and speakers’ gender on the recognition of vocal emotions. They stress the importance of distinguishing these factors to explain recognition ability in the processing of emotional prosody. (shrink)

Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition. Based on machine learning, such technology can tremendously increase the amount of processed data. FER is now easily accessible and has been validated for the classification of standardized prototypical facial expressions. However, applicability to more naturalistic facial expressions still remains uncertain. Hence, we test and compare performance of three different FER systems with human emotion (...) class='Hi'>recognition for standardized posed facial expressions and for non-standardized acted facial expressions. For the standardized images, all three systems classify basic emotions accurately and they are mostly on par with human raters. For the non-standardized stimuli, performance drops remarkably for all three systems, but Azure still performs similarly to humans. In addition, all systems and humans alike tend to misclassify some of the non-standardized emotional facial expressions as neutral. In sum, emotion recognition by automated facial expression recognition can be an attractive alternative to human emotion recognition for standardized and non-standardized emotional facial expressions. However, we also found limitations in accuracy for specific facial expressions; clearly there is need for thorough empirical evaluation to guide future developments in computer vision of emotional facial expressions. (shrink)

There are considerable gaps in our knowledge of how children develop abstract language. In this paper, we tested the Affective Embodiment Account, which proposes that emotional information is more essential for abstract than concrete conceptual development. We tested the recognition memory of 7- and 8-year-old children, as well as a group of adults, for abstract and concrete words which differed categorically in valence. Word valence significantly interacted with concreteness in hit rates of both children and adults, such that effects (...) of valence were only found in memory for abstract words. The pattern of valence effects differed for children and adults: children remembered negative words more accurately than neutral and positive words, whereas adults remembered negative and positive words more accurately than neutral words. In addition, signal detection analysis revealed that children were better able to discriminate negative than positive words, regardless of concreteness. The findings suggest that the memory accuracy of 7- and 8-year-old children is influenced by emotional information, particularly for abstract words. The results are in agreement with the Affective Embodiment Account and with multimodal accounts of children’s lexical development. (shrink)

An ability to accurately recognize negative emotions in others can initiate pro-social behavior and prevent anti-social actions. Thus, it remains of an interest of scholars studying effects of violent video games. While exposure to such games was linked to slower emotion recognition, the evidence regarding accuracy of emotion recognition among players of violent games is weak and inconsistent. The present research investigated the relationship between violent video game exposure and accuracy of negative emotion (...) recognition. We assessed the level of self-reported VVGE in hours per day and the accuracy of the recognition using the Facial Expressions Matching Test. The results, with adolescents and with adults, showed that VVGE was negatively related to accurate recognition of negative emotion expressions, even if controlled for age, gender, and trait empathy, but no causal direction could be assessed. In line with the violent media desensitization model, our findings suggest that higher self-reported VVGE relates to lower recognition of negative emotional expressions of other people. On the one hand, such lower recognition of negative emotions may underlie inaccurate reactions in real-life social situations. On the other hand, lower sensitivity to social cues may help players to better focus on their performance in a violent game. (shrink)

ABSTRACTThe recognition of emotional facial expressions is often subject to contextual influence, particularly when the face and the context convey similar emotions. We investigated whether spontaneous, incidental affective theory of mind inferences made while reading vignettes describing social situations would produce context effects on the identification of same-valenced emotions as well as differently-valenced emotions conveyed by subsequently presented faces. Crucially, we found an effect of context on reaction times in both experiments while, in line with previous work, we found (...) evidence for a context effect on accuracy only in Experiment 1. This demonstrates that affective theory of mind inferences made at the pragmatic level of a text can automatically, contextually influence the perceptual processing of emotional facial expressions in a separate task even when those emotions are of a distinctive valence. Thus, our novel findings suggest that language acts as a contex... (shrink)

Research on facial emotion expression has mostly focused on emotion recognition, assuming that a small number of discrete emotions is elicited and expressed via prototypical facial muscle configurations as captured in still photographs. These are expected to be recognized by observers, presumably via template matching. In contrast, appraisal theories of emotion propose a more dynamic approach, suggesting that specific elements of facial expressions are directly produced by the result of certain appraisals and predicting the facial patterns (...) to be expected for certain appraisal configurations. This approach has recently been extended to emotion perception, claiming that observers first infer individual appraisals and only then make categorical emotion judgments based on the estimated appraisal patterns, using semantic inference rules. Here we report two related studies to empirically investigate the facial action unit configurations that are used by actors to convey specific emotions in short affect bursts and to examine to what extent observers can infer a person’s emotions from the predicted facial expression configurations. The results show that 1) professional actors use many of the predicted facial action unit patterns to enact systematically specified appraisal outcomes in a realistic scenario setting, and 2) naïve observers infer the respective emotions based on highly similar facial movement configurations with a degree of accuracy comparable to earlier research findings. Based on estimates of underlying appraisal criteria for the different emotions we conclude that the patterns of facial action units identified in this research correspond largely to prior predictions and encourage further research on appraisal-driven expression and inference. (shrink)

What would it mean for an emotion to successfully “recognize” something about an object toward which it is directed? Although the notion of "emotional recognition" is central to Rick Furtak’s _Knowing Emotions_, the text does not provide an account of this concept that enables us to assess the extent to which a given emotional response is recognitive. This article draws from the text to articulate a novel account of emotional recognition. According to this account, emotional recognition (...) can be assessed not only in terms of the “accuracy” of an emotional construal in a strictly epistemological sense, but also in terms of the quasi-ethical ideal of responding emotionally to what we encounter in ways that are “specific,” “deep," and “balanced.". (shrink)

Recognising and responding appropriately to emotions is critical to adaptive psychological functioning. Psychopathic traits (e.g. callous, manipulative, impulsive, antisocial) are related to differences in recognition and response when emotion is conveyed through facial expressions and language. Use of emotional music stimuli represents a promising approach to improve our understanding of the specific emotion processing difficulties underlying psychopathic traits because it decouples recognition of emotion from cues directly conveyed by other people (e.g. facial signals). In Experiment (...) 1, participants listened to clips of emotional music and identified the emotional content (Sample 1, N = 196) or reported on their feelings elicited by the music (Sample 2, N = 197). Participants accurately recognised (t(195) = 32.78, p <.001, d = 4.69) and reported feelings consistent with (t(196) = 7.84, p <.001, d = 1.12) the emotion conveyed in the music. However, psychopathic traits were associated with reduced emotion recognition accuracy (F(1, 191) = 19.39, p <.001) and reduced likelihood of feeling the emotion (F(1, 193) = 35.45, p <.001), particularly for fearful music. In Experiment 2, we replicated findings for broad difficulties with emotion recognition (Sample 3, N = 179) and emotional resonance (Sample 4, N = 199) associated with psychopathic traits. Results offer new insight into emotion recognition and response difficulties that are associated with psychopathic traits. (shrink)

ABSTRACTPosed stimuli dominate the study of nonverbal communication of emotion, but concerns have been raised that the use of posed stimuli may inflate recognition accuracy relative to spontaneous expressions. Here, we compare recognition of emotions from spontaneous expressions with that of matched posed stimuli. Participants made forced-choice judgments about the expressed emotion and whether the expression was spontaneous, and rated expressions on intensity and prototypicality. Listeners were able to accurately infer emotions from both posed and (...) spontaneous expressions, from auditory, visual, and audiovisual cues. Furthermore, perceived intensity and prototypicality were found to play a role in the accurate recognition of emotion, particularly from spontaneous expressions. Our findings demonstrate that perceivers can reliably recognise emotions from spontaneous expressions, and that depending on the comparison set, recognition levels can even be equivalent to that o... (shrink)

Emotional recognition has arisen as an essential field of study that can expose a variety of valuable inputs. Emotion can be articulated in several means that can be seen, like speech and facial expressions, written text, and gestures. Emotion recognition in a text document is fundamentally a content-based classification issue, including notions from natural language processing (NLP) and deep learning fields. Hence, in this study, deep learning assisted semantic text analysis (DLSTA) has been proposed for human (...) emotion detection using big data. Emotion detection from textual sources can be done utilizing notions of Natural Language Processing. Word embeddings are extensively utilized for several NLP tasks, like machine translation, sentiment analysis, and question answering. NLP techniques improve the performance of learning-based methods by incorporating the semantic and syntactic features of the text. The numerical outcomes demonstrate that the suggested method achieves an expressively superior quality of human emotion detection rate of 97.22% and the classification accuracy rate of 98.02% with different state-of-the-art methods and can be enhanced by other emotional word embeddings. (shrink)

People dedicate significant attention to others’ facial expressions and to deciphering their meaning. Hence, knowing whether such expressions are genuine or deliberate is important. Early research proposed that authenticity could be discerned based on reliable facial muscle activations unique to genuine emotional experiences that are impossible to produce voluntarily. With an increasing body of research, such claims may no longer hold up to empirical scrutiny. In this article, expression authenticity is considered within the context of senders’ ability to produce convincing (...) facial displays that resemble genuine affect and human decoders’ judgments of expression authenticity. This includes a discussion of spontaneous vs. posed expressions, as well as appearance- vs. elicitation-based approaches for defining emotion recognition accuracy. We further expand on the functional role of facial displays as neurophysiological states and communicative signals, thereby drawing upon the encoding-decoding and affect-induction perspectives of emotion expressions. Theoretical and methodological issues are addressed with the aim to instigate greater conceptual and operational clarity in future investigations of expression authenticity. (shrink)

Android robots capable of emotional interactions with humans have considerable potential for application to research. While several studies developed androids that can exhibit human-like emotional facial expressions, few have empirically validated androids’ facial expressions. To investigate this issue, we developed an android head called Nikola based on human psychology and conducted three studies to test the validity of its facial expressions. In Study 1, Nikola produced single facial actions, which were evaluated in accordance with the Facial Action Coding System. The (...) results showed that 17 action units were appropriately produced. In Study 2, Nikola produced the prototypical facial expressions for six basic emotions, and naïve participants labeled photographs of the expressions. The recognition accuracy of all emotions was higher than chance level. In Study 3, Nikola produced dynamic facial expressions for six basic emotions at four different speeds, and naïve participants evaluated the naturalness of the speed of each expression. The effect of speed differed across emotions, as in previous studies of human expressions. These data validate the spatial and temporal patterns of Nikola’s emotional facial expressions, and suggest that it may be useful for future psychological studies and real-life applications. (shrink)

People are ingesting various information from different sense organs all the time to complete different cognitive tasks. The brain integrates and regulates this information. The two significant sensory channels for receiving external information are sight and hearing that have received extensive attention. This paper mainly studies the effect of music and visual-auditory stimulation on electroencephalogram of happy emotion recognition based on a complex system. In the experiment, the presentation was used to prepare the experimental stimulation program, and the (...) cognitive neuroscience experimental paradigm of EEG evoked by happy emotion pictures was established. Using 93 videos as natural stimuli, fMRI data were collected. Finally, the collected EEG signals were removed with the eye artifact and baseline drift, and the t-test was used to analyze the significant differences of different lead EEG data. Experimental data shows that, by adjusting the parameters of the convolutional neural network, the highest accuracy of the two-classification algorithm can reach 98.8%, and the average accuracy can reach 83.45%. The results show that the brain source under the combined visual and auditory stimulus is not a simple superposition of the brain source of the single visual and auditory stimulation, but a new interactive source is generated. (shrink)

Emotional design is an important development trend of interaction design. Emotional design in products plays a key role in enhancing user experience and inducing user emotional resonance. In recent years, based on the user's emotional experience, the design concept of strengthening product emotional design has become a new direction for most designers to improve their design thinking. In the emotional interaction design, the machine needs to capture the user's key information in real time, recognize the user's emotional state, and use (...) a variety of clues to finally determine the appropriate user model. Based on this background, this research uses a deep learning mechanism for more accurate and effective emotion recognition, thereby optimizing the design of the interactive system and improving the user experience. First of all, this research discusses how to use user characteristics such as speech, facial expression, video, heartbeat, etc., to make machines more accurately recognize human emotions. Through the analysis of various characteristics, the speech is selected as the experimental material. Second, a speech-based emotion recognition method is proposed. The mel-Frequency cepstral coefficient of the speech signal is used as the input of the improved long and short-term memory network. To ensure the integrity of the information and the accuracy of the output at the next moment, ILSTM makes peephole connections in the forget gate and input gate of LSTM, and adds the unit state as input data to the threshold layer. The emotional features obtained by ILSTM are input into the attention layer, and the self-attention mechanism is used to calculate the weight of each frame of speech signal. The speech features with higher weights are used to distinguish different emotions and complete the emotion recognition of the speech signal. Experiments on the EMO-DB and CASIA datasets verify the effectiveness of the model for emotion recognition. Finally, the feasibility of emotional interaction system design is discussed. (shrink)

Congenital amusia in its most common form is a disorder characterized by a musical pitch processing deficit. Although pitch is involved in conveying emotion in music, the implications for pitch deficits on musical emotion judgements is still under debate. Relatedly, both limited and spared musical emotion recognition was reported in amusia in conditions where emotion cues were not determined by musical mode or dissonance. Additionally, assumed links between musical abilities and visuo-spatial attention processes need further (...) investigation in congenital amusics. Hence, we here test to what extent musical emotions can influence attentional performance. Fifteen congenital amusic adults and fifteen healthy controls matched for age and education were assessed in three attentional conditions: executive control, alerting, and orienting while music expressing either joy, tenderness, sadness, or tension was presented. Visual target detection was in the normal range for both accuracy and response times in the amusic relative to the control participants. Moreover, in both groups, music exposure produced facilitating effects on selective attention that appeared to be driven by the arousal dimension of musical emotional content, with faster correct target detection during joyful compared to sad music. These findings corroborate the idea that pitch processing deficits related to congenital amusia do not impede other cognitive domains, particularly visual attention. Furthermore, our study uncovers an intact influence of music and its emotional content on the attentional abilities of amusic individuals. The results highlight the domain-selectivity of the pitch disorder in congenital amusia, which largely spares the development of visual attention and affective systems. (shrink)

Electroencephalogram has been widely utilized in emotion recognition. Psychologists have found that emotions can be divided into conscious emotion and unconscious emotion. In this article, we explore to classify subliminal emotions with EEG signals elicited by subliminal face stimulation, that is to select appropriate features to classify subliminal emotions. First, multi-scale sample entropy, wavelet packet energy, and wavelet packet entropy of EEG signals are extracted. Then, these features are fed into the decision tree and improved random (...) forest, respectively. The classification accuracy with Ei and WpEn is higher than MSpEn, which shows that Ei and WpEn can be used as effective features to classify subliminal emotions. We compared the classification results of different features combined with the decision tree algorithm and the improved random forest algorithm. The experimental results indicate that the improved random forest algorithm attains the best classification accuracy for subliminal emotions. Finally, subliminal emotions and physiological proof of subliminal affective priming effect are discussed. (shrink)

Electroencephalography is the measurement of neuronal activity in different areas of the brain through the use of electrodes. As EEG signal technology has matured over the years, it has been applied in various methods to EEG emotion recognition, most significantly including the use of convolutional neural network. However, these methods are still not ideal, and shortcomings have been found in the results of some models of EEG feature extraction and classification. In this study, two CNN models were selected (...) for the extraction and classification of preprocessed data, namely, common spatial patterns- CNN and wavelet transform- CNN. Using the CSP-CNN, we first used the common space model to reduce dimensionality and then applied the CNN directly to extract and classify the features of the EEG; while, with the WT-CNN model, we used the wavelet transform to extract EEG features, thereafter applying the CNN for classification. The EEG classification results of these two classification models were subsequently analyzed and compared, with the average classification accuracy of the CSP-CNN model found to be 80.56%, and the average classification accuracy of the WT-CNN model measured to 86.90%. Thus, the findings of this study show that the average classification accuracy of the WT-CNN model was 6.34% higher than that of the CSP-CNN. (shrink)

Processing and recognizing facial expressions are key factors in human social interaction. Past research suggests that individuals with autism spectrum disorder present difficulties to decode facial expressions. Those difficulties are notably attributed to altered strategies in the visual scanning of expressive faces. Numerous studies have demonstrated the multiple benefits of exposure to pet dogs and service dogs on the interaction skills and psychosocial development of children with ASD. However, no study has investigated if those benefits also extend to the processing (...) of facial expressions. The aim of this study was to investigate if having a service dog had an influence on facial expression processing skills of children with ASD. Two groups of 15 children with ASD, with and without a service dog, were compared using a facial expression recognition computer task while their ocular movements were measured using an eye-tracker. While the two groups did not differ in their accuracy and reaction time, results highlighted that children with ASD owning a service dog directed less attention toward areas that were not relevant to facial expression processing. They also displayed a more differentiated scanning of relevant facial features according to the displayed emotion. Results from the present study suggest that having a service dog and interacting with it on a daily basis may promote the development of specific visual exploration strategies for the processing of human faces. (shrink)

Micro-expressions, as fleeting facial expressions, are very important for judging people’s true emotions, thus can provide an essential behavioral clue for lie and dangerous demeanor detection. From embodied accounts of cognition, we derived a novel hypothesis that facial feedback from upper and lower facial regions has differential effects on micro-expression recognition. This hypothesis was tested and supported across three studies. Specifically, the results of Study 1 showed that people became better judges of intense micro-expressions with a duration of 450 (...) ms when the facial feedback from upper face was enhanced via a restricting gel. Additional results of Study 2 showed that the recognition accuracy of subtle micro-expressions was significantly impaired under all duration conditions (50 ms, 150 ms, 333 ms, and 450 ms) when facial feedback from lower face was enhanced. In addition, the results of Study 3 also revealed that blocking the facial feedback of lower face, significantly boosted the recognition accuracy of subtle and intense micro-expressions under all duration conditions (150 ms and 450 ms). Together, these results highlight the role of facial feedback in judging the subtle movements of micro-expressions. (shrink)

Education public opinion information management is an important research focus in the field of Education Data Mining. In this paper, we classify the education data information based on the traditional Flat-OCC model. From the cognitive psychology perspective, we identify up to 12 kinds of emotions, including sadness and happiness. In addition, the EMO-CBOW model is also proposed in this paper to further identify emotion by using various emoticons in educational data sets. The empirical result shows that the proposed Flat-OCC (...) model can classify and identify the emotion of education public opinion data well; and for the recognition of educational emotion classification, the categorization accuracy of the Flat-OCC+EMO-CBOW model is significantly higher than that of a single Flat-OCC model, which reveals that the emotional-pack-based model we propose can enhance our benchmark model. (shrink)

The COVID-19 pandemic has introduced new challenges for governments and individuals. Unprecedented efforts at reducing virus transmission launched a novel arena for human face recognition in which faces are partially occluded with masks. Previous studies have shown that masks decrease accuracy of face identity and emotion recognition. The current study focuses on the impact of masks on the speed of processing of these and other important social dimensions. Here we provide a systematic assessment of the impact (...) of COVID-19 masks on facial identity, emotion, gender, and age. Four experiments were conducted in which participants categorized faces on a predefined dimension. Both speed and accuracy were measured. The results revealed that masks hindered the perception of virtually all tested facial dimensions, interfering with normal speed and accuracy of categorization. We also found that the unwarranted effects of masks were not due to holistic processes, because the Face Inversion Effect was generally not larger with unmasked compared with masked faces. Moreover, we found that the impact of masks is not automatic and that under some contexts observers can control at least part of their detrimental effects. (shrink)

Previous research has shown that the proactive deprioritization of emotional distractors through the provision of information about the distractors or passive habituation of emotional distractors may attenuate emotion-induced blindness (EIB) in the rapid serial visual presentation stream. However, whether prior memory encoding of emotional distractors could bias the EIB effect remains unknown. To address this question, this study employed a three-phase paradigm integrating an item-method direct forgetting (DF) procedure with a classic EIB procedure. Participants completed a memory coding phase (...) to either remember or forget negative pictures, then performed an intermediate phase of the EIB test, and finally finished a recognition test. Critically, the same to-be-forgotten (TBF) and to-be-remembered (TBR) negative pictures in the memory learning phase were used as emotional distractors in the intermediate EIB test. The results replicated the typical DF effect by showing higher recognition accuracies for TBR pictures compared to those for TBF pictures. More importantly, the TBF negative distractors attenuated the EIB effect compared to the TBR negative distractors, but showed a comparable EIB effect as the novel negative distractors. These findings indicate that prior memory encoding manipulations of negative distractors could bias subsequent EIB effects, providing an important approach to modulate the EIB effect. (shrink)

The production of emotional speech is determined by the movement of the speaker’s tongue, lips, and jaw. In order to combine articulatory data and acoustic data of speakers, articulatory-to-acoustic conversion of emotional speech has been studied. In this paper, parameters of LSSVM model have been optimized using the PSO method, and the optimized PSO-LSSVM model was applied to the articulatory-to-acoustic conversion. The root mean square error and mean Mel-cepstral distortion have been used to evaluate the results of conversion; the evaluated (...) result illustrates that MMCD of MFCC is 1.508 dB, and RMSE of the second formant is 25.10 Hz. The results of this research can be further applied to the feature fusion of emotion speech recognition to improve the accuracy of emotion recognition. (shrink)

Background: Clinical and experimental findings have suggested that insomnia is associated with altered emotion processing, such as facial emotion recognition and impairments in executive functions. However, the results still appear non-consensual and have recently been presented by a few number of studies. Accordingly, the aim of the present study was to investigate whether patients with Insomnia disorder will present alterations in recognition of facial emotions and that such alterations will be related to Executive Functions and that (...) Insomnia Disorder patients will exhibit a lower Executive Functions performance when compared to normal controls individuals. Method: A total of 11 individuals diagnosed with insomnia disorder and 15 control participants took part in the present cross-sectional study. They completed questionnaires covering socio-demographic data, a sleep and mood evaluation, and a cognitive assessment. Moreover, they underwent a computerized Facial Emotional Recognition test, consisting of facial emotion labeling and facial emotion matching. Results: Individuals with insomnia presented a lower recognition of facial expressions of fear and sadness in comparison to control volunteers. Perceptual Organization Index was found to be statistically significant lower in the insomnia group compared to the control group. There was a statistically significant association between facial recognition emotional and cognitive processes of in executive functions performance, especially in processes related to the inhibitory control, planning capacity, problem solving and cognitive flexibility in insomnia group. Conclusions: Insomnia disorder impairs the facial emotion recognition of the fear and sadness. Insomniacs with impairment in the facial emotion recognition had impaired performance of executive functions. There is a negative impact on processes related to cognitive-emotional capacity than on processes related to social-emotional recognition. Keywords: insomnia, facial emotional recognition, sleep deprivation, cognition, executive functions. (shrink)

This study mainly focuses on the emotion analysis method in the application of psychoanalysis based on sentiment recognition. The method is applied to the sentiment recognition module in the server, and the sentiment recognition function is effectively realized through the improved convolutional neural network and bidirectional long short-term memory model. First, the implementation difficulties of the C-BiL model and specific sentiment classification design are described. Then, the specific design process of the C-BiL model is introduced, and (...) the innovation of the C-BiL model is indicated. Finally, the experimental results of the models are compared and analyzed. Among the deep learning models, the accuracy of the C-BiL model designed in this study is relatively high irrespective of the binary classification, the three classification, or the five classification, with an average improvement of 2.47% in Diary data set, 2.16% in Weibo data set, and 2.08% in Fudan data set. Therefore, the C-BiL model designed in this study can not only successfully classify texts but also effectively improve the accuracy of text sentiment recognition. (shrink)

Metaverse is to build a virtual world that is both mapped and independent of the real world in cyberspace by using the improvement in the maturity of various digital technologies, such as virtual reality, augmented reality, big data, and 5G, which is important for the future development of a wide variety of professions, including education. The metaverse represents the latest stage of the development of visual immersion technology. Its essence is an online digital space parallel to the real world, which (...) is becoming a practical field for the innovation and development of human society. The most prominent advantage of the English-teaching metaverse is that it can provide an immersive and interactive teaching field for teachers and students, simultaneously meeting the teaching and learning needs of teachers and students in both the physical world and virtual world. This study constructs experiential situational English-teaching scenario and convolutional neural networks –recurrent neural networks fusion models are proposed to recognize students’ emotion electroencephalogram in experiential English teaching during the feature space of time domain, frequency domain, and spatial domain. Analyzing EEG data collected by OpenBCI EEG Electrode Cap Kit from students, experiential English-teaching scenario is designed into three types: sequential guidance, comprehensive exploration, and crowd-creation construction. Experimental data analysis of the three kinds of learning activities shows that metaverse-powered experiential situational English teaching can promote the improvement of students’ sense of interactivity, immersion, and cognition, and the accuracy and analysis time of CNN–RNN fusion model is much higher than that of baselines. This study can provide a nice reference for the emotion recognition of students under COVID-19. (shrink)

This chapter argues that emotion recognition is a skill. A skill perspective on emotion recognition draws attention to underappreciated features of this cornerstone of social cognition. Skills have a number of characteristic features. For example, they are improvable, practical, and flexible. Emotion recognition has these features as well. Leading theories of emotion recognition often draw inadequate attention to these features. The chapter advances a theory of emotion recognition that is better (...) suited to this purpose. It proposes that emotion recognition involves scripts. Emotion scripts describe how people are likely to behave in different emotional contexts. Scripts can be improved with the addition of richer social knowledge, they are practical in that they describe and guide social interaction, and they are flexible because they must accommodate the fact that emotions have different behavioral impact depending on the agents involved and the circumstances at play. Learning to recognize emotions through scripts qualifies as a skill. (shrink)

Fraud can cause severe financial losses and affect the physical and mental health of victims. This study aimed to explore the manipulative characteristics of fraudsters and their relationship with other psychological variables. Thirty-four fraudsters were selected from a medium-security prison in China, and thirty-one healthy participants were recruited online. Both groups completed an emotional face-recognition task and self-report measures assaying emotional manipulation, psychopathy, emotion recognition, and empathy. Results showed that imprisoned fraudsters had higher accuracy in identifying (...) fear and surprise faces but lower accuracy in identifying happiness than controls. Significantly lower scores on non-prosocial factors on the Managing the Emotions of Others scale were found for imprisoned fraudsters, relative to controls. Imprisoned fraudsters had low scores in the assessment of psychopathy than the control group, especially Factor 1. For empathy, imprisoned fraudsters had significantly higher scores in perspective-taking than controls. Correlation analyses revealed that psychopathic traits were positively correlated with non-prosocial factors in both groups. However, the relationships between emotional manipulation and emotional recognition and empathy were not consistent across the groups. The results suggest that fraudsters may pretend to be as prosocial as healthy controls, who had lower antisocial tendencies, normal empathy ability, and would like to manipulate others’ emotions positively during social interaction. (shrink)

How to enable the computer to accurately analyze the emotional information and story background of characters in Qin opera is a problem that needs to be studied. To promote the artistic inheritance and cultural emotion color dissemination of Qin opera, an emotion analysis model of Qin opera based on attention residual network is presented. The neural network is improved and optimized from the perspective of the model, learning rate, network layers, and the network itself, and then multi-head attention (...) is added to the ResNet to increase the recognition ability of the model. The convolutional neural network is optimized from the internal depth, and the fitting ability and stability of the model are enhanced through the ResNet model. Combined with the attention mechanism, the expression of each weight information is strengthened. The multi-head attention mechanism is introduced in the model and a multi-head attention ResNet, namely, MHAtt_ResNet, is proposed. The network structure can effectively identify the features of the spectrogram, improve the weight information of spectrogram features, and deepen the relationship between distant information in long-time series. Through experiments, the proposed model has high emotional classification accuracy for Qin opera, and with the increase of the number of data sets, the model will train a better classification effect. (shrink)

Prior studies have reported that women outperform men in nonverbal communication, including the recognition of emotions through static facial expressions. In this experimental study, we investigated sex differences in the estimation of states of happiness, anger, fear, and disgust through static photographs using a two-culture approach. This study was conducted among the Tuvans and Mongolian people from Southern Siberia. The respondents were presented with a set of photographs of men and women of European and Tuvan origin and were asked (...) to interpret each of them. They were asked: “What does the person in the photo feel?” We found that the Tuvans easily identified happiness and anger; however, the level of accuracy of fear and disgust recognition was low. No sex differences in the recognition of happiness, disgust, and fear were observed. However, anger recognition was significantly moderated by the perceiver’s sex and the origin of the model. Compared to Tuvan men, Tuvan women were significantly less accurate in identifying anger in male Tuvans. Furthermore, the age effect was found in recognition of fear: older Tuvans were more accurate while recognizing the fearful faces of Tuvan, but not the European models. (shrink)

The electroencephalogram is the most common method used to study emotions and capture electrical brain activity changes. Long short-term memory processes the temporal characteristics of data and is mostly used for emotional text and speech recognition. Since an EEG involves a time series signal, this article mainly studied the introduction of LSTM for emotional EEG recognition. First, an ALL-LSTM model with a four-layered LSTM network was established in which the average accuracy rate for emotional classification reached 86.48%. (...) Second, four EEG characteristics were extracted via the wavelet transform using the LSTM-based sentiment classification network. The experimental results showed that the best average classification accuracy of these four features was 73.48%. This was 13% lower than in the ALL-LSTM model, indicating that inappropriate feature extraction methods could destroy the timing of EEG signals. LSTM can be used to thoroughly examine EEG signal timing and preprocessed EEG data. The accuracy and stability of the ALL-LSTM model are significantly superior to those of the WT-LSTM model. The result showed that the process of emotion generation based on EEG is sequential. Compared with EEG emotion extraction using WT, the raw EEG signal’s timing is more suitable for the LSTM network. (shrink)

Previous meta-analyses support a female advantage in decoding non-verbal emotion (Hall, 1978, 1984), yet the mechanisms underlying this advantage are not understood. The present study examined whether the female advantage is related to greater female attention to the eyes. Eye-tracking techniques were used to measure attention to the eyes in 19 males and 20 females during a facial expression recognition task. Women were faster and more accurate in their expression recognition compared with men, and women looked more (...) at the eyes than men. Positive relationships were observed between dwell time and number of fixations to the eyes and both accuracy of facial expression recognition and speed of facial expression recognition. These results support the hypothesis that the female advantage in facial expression recognition is related to greater female attention to the eyes. (shrink)

We develop a version of a direct perception account of emotion recognition on the basis of a metaphysical claim that emotions are individuated as patterns of characteristic features. On our account, emotion recognition relies on the same type of pattern recognition as is described for object recognition. The analogy allows us to distinguish two forms of directly perceiving emotions, namely perceiving an emotion in the absence of any top-down processes, and perceiving an (...) class='Hi'>emotion in a way that significantly involves some top-down processes ; and, in addition, an inference-based evaluation of an emotion. Our model clarifies the epistemology of emotion recognition. (shrink)