Reusing Neural Speech Representations for Auditory Emotion Recognition

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 423–430, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Reusing Neural Speech Representations for Auditory Emotion Recognition Egor Lakomkin Cornelius Weber Sven Magg Stefan Wermter Department of Informatics, Knowledge Technology University of Hamburg Vogt-Koelln Str. 30, 22527 Hamburg, Germany {lakomkin, weber, magg, wermter}@informatik.uni-hamburg.de Abstract Acoustic emotion recognition aims to cat- egorize the affective state of the speaker and is still a difficult task for machine learning models. The difficulties come from the scarcity of training data, general subjectivity in emotion perception result- ing in low annotator agreement, and the uncertainty about which features are the most relevant and robust ones for clas- sification. In this paper, we will tackle the latter problem. Inspired by the re- cent success of transfer learning methods we propose a set of architectures which utilize neural representations inferred by training on large speech databases for the acoustic emotion recognition task. Our ex- periments on the IEMOCAP dataset show 10% relative improvements in the accu- racy and F1-score over the baseline recur- rent neural network which is trained end- to-end for emotion recognition. 1 Introduction Speech emotion recognition (SER) has received growing interest and attention in recent years. Be- ing able to predict the affective state of a person gives valuable information which could improve dialog systems in human-computer interaction. To fully understand a current emotion expressed by a person, also knowledge of the context is required, like facial expressions, the semantics of a spoken text, gestures and body language, and cultural pe- culiarities. This makes it challenging even for peo- ple that have all this information to accurately pre- dict the affective state. In this work, we are fo- cusing solely on inferring the speaker’s emotional state by analysing acoustic signals which are also the only source of information in situations when Figure 1: High-level system architecture. Acous- tic emotion recognition system uses neural speech representations for affective state classification. the speaker is not directly observable. With recent advances in deep learning, which made it possible to train large end-to-end mod- els for image classification (Simonyan and Zis- serman, 2014), speech recognition (Hannun et al., 2014) and natural language understanding (Sutskever et al., 2014), the majority of the current work in the area of acoustic emotion recognition is neural network-based. Diverse neural architec- tures were investigated based on convolutional and recurrent neural networks (Fayek et al., 2017; Tri- georgis et al., 2016). Alternatively, methods based on linear models, like SVM with careful feature engineering, still show competitive performance on the benchmark datasets (Schuller et al., 2013, 2009). Such methods were popular in computer vision until the AlexNet approach (Krizhevsky et al., 2012) made automatic feature learning more wide-spread. The low availability of annotated au- ditory emotion data is probably one of the main reasons for traditional methods being competitive. The appealing property of neural networks com- pared to SVM-like methods is their ability to iden- tify automatically useful patterns in …

Content truncated to 500 words