Accuracy of Automatic Cross-Corpus Emotion Labeling for Conversational Speech Corpus Commonization

Accuracy of Automatic Cross-Corpus Emotion Labeling for Conversational Speech Corpus Commonization Hiroki Mori†, Atsushi Nagaoka†, Yoshiko Arimoto‡§ †Graduate School of Engineering, Utsunomiya University, Japan ‡Brain Science Institute, RIKEN, Japan §College of Engineering, Shibaura Institute of Technology, Japan {hiroki,atusi}@speech-lab.org, ar@brain.riken.jp Abstract There exists a major incompatibility in emotion labeling framework among emotional speech corpora, that is, category-based and dimension-based. Commonizing these requires inter-corpus emotion labeling according to both frameworks, but doing this by hu- man annotators is too costly for most cases. This paper examines the possibility of automatic cross-corpus emotion labeling. In order to evaluate the effectiveness of the automatic labeling, a comprehensive emotion annotation for two conversational corpora, UUDB and OGVC, was performed. With a state-of-the-art machine learning technique, dimensional and categorical emotion estimation models were trained and tested against the two corpora. For the emotion dimension estimation, the automatic cross-corpus emotion labeling for the different corpus was effective for the dimensions of aroused-sleepy, dominant-submissive and interested-indifferent, showing only slight performance degradation against the result for the same corpus. On the other hand, the performance for the emotion category estimation was not sufficient. Keywords: emotion, spontaneous speech, emotion recognition, openSMILE, support vector machine 1. Introduction Large-scale spoken language resources are essential for un- derstanding and modeling speech and other human behav- iors. A wide variety of corpus-based speech technologies have been developed for recognizing, reproducing or ma- nipulating verbal and nonverbal behaviors in human com- munication. Among them, affective computing is one of the most prominent research trends. Research goals of af- fective computing include analysis and modeling of speech emotion, recognition or detection of the speaker’s affective state (Schuller et al., 2013), and emotional speech synthesis (Nagata et al., 2013). Although several speech corpora have been developed for studying speech emotion, it is widely recognized that less acted, more realistic data are needed (Schuller et al., 2013). However, developing a naturalistic emotional corpus tends to suffer from the problem of scalability, mainly due to the costly work of annotating emotion. If multiple emotional corpora could be put together into a large corpus, it would benefit many studies and appli- cations. However, the framework used by each corpus for describing emotion is generally incompatible with each other. A major incompatibility comes from the theory of emotion assumed, i.e. category-based and dimension-based (Cowie and Cornelius, 2003). The former assumes some emotion categories such as the “Big Six” emotions (fear, anger, happiness, sadness, surprise, disgust), while the lat- ter does not assume discrete categories but rather regards an emotional state as a point in a space of a small number of dimensions. Emotional speech corpora with category- based labels include Berlin Database of Emotional Speech (Emo-DB) (Burkhardt et al., 2005), Parameterized & An- notated CMU Let’s Go (LEGO) Database (Schmitt et al., 2012), Surrey Audio-Visual Expressed Emotion (SAVEE) Database (Haq and Jackson, 2010), FAU Aibo Emotion Corpus (Steidl, 2009) and Online Gaming Voice Chat Cor- pus with Emotional Label (OGVC) (Arimoto et al., 2012), while those with dimension-based labels include Vera …

Content truncated to 500 words