Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15681–15695 July 9-14, 2023 ©2023 Association for Computational Linguistics Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression Wen Wu1, Chao Zhang2, Philip C. Woodland1 1 Department of Engineering, University of Cambridge, Cambridge, UK 2 Department of Electrical Engineering, Tsinghua University, Beijing, China {ww368, pcw}@eng.cam.ac.uk; cz277@tsinghua.edu.cn Abstract In automatic emotion recognition (AER), la- bels assigned by different human annotators to the same utterance are often inconsistent due to the inherent complexity of emotion and the subjectivity of perception. Though determin- istic labels generated by averaging or voting are often used as the ground truth, it ignores the intrinsic uncertainty revealed by the incon- sistent labels. This paper proposes a Bayesian approach, deep evidential emotion regression (DEER), to estimate the uncertainty in emotion attributes. Treating the emotion attribute la- bels of an utterance as samples drawn from an unknown Gaussian distribution, DEER places an utterance-specific normal-inverse gamma prior over the Gaussian likelihood and predicts its hyper-parameters using a deep neural net- work model. It enables a joint estimation of emotion attributes along with the aleatoric and epistemic uncertainties. AER experiments on the widely used MSP-Podcast and IEMOCAP datasets showed DEER produced state-of-the- art results for both the mean values and the distribution of emotion attributes1. 1 Introduction Automatic emotion recognition (AER) is the task that enables computers to predict human emotional states based on multimodal signals, such as au- dio, video and text. An emotional state is de- fined based on either categorical or dimensional theory. The categorical theory claims the exis- tence of a small number of basic discrete emo- tions ( i.e. anger and happy) that are inherent in our brain and universally recognised (Gunes et al., 2011; Plutchik, 2001). Dimensional emotion the- ory characterises emotional states by a small num- ber of roughly orthogonal fundamental continuous- valued bipolar dimensions (Schlosberg, 1954; Nico- laou et al., 2011) such as valence-arousal and ap- proach–avoidance (Russell and Mehrabian, 1977; 1Code available: https://github.com/W-Wu/DEER Russell, 1980; Grimm et al., 2007). These dimen- sions are also known as emotion attributes, which allow us to model more subtle and complex emo- tions and are thus more common in psychological studies. As a result, AER includes a classification approach based on emotion-class-based labels and a regression approach based on attribute-based la- bels. This paper focuses on attribute-based AER with speech input. Emotion annotation is challenging due to the in- herent ambiguity of mixed emotion, the personal variations in emotion expression, the subjectivity in emotion perception, etc. Most AER datasets use multiple human annotators to label each utterance, which often results in inconsistent labels, either as emotion categories or attributes. This is also a typi- cal manifestation of the intrinsic data uncertainty, also referred to as aleatoric uncertainty (Matthies, 2007; Der Kiureghian and Ditlevsen, 2009), that arises from the natural complexity of emotion data. It is common to replace such inconsistent labels with deterministic labels obtained by majority vot- …

Content truncated to 500 words