Could Speaker, Gender or Age Awareness be beneficial in Speech-based Emotion Recognition?

Could Speaker, Gender or Age Awareness be beneficial in Speech-based Emotion Recognition? Maxim Sidorov1, Alexander Schmitt1, Eugene Semenkin2 and Wolfgang Minker1 1Ulm University, 2Siberian State Aerospace University Ulm Germany, Krasnoyarsk Russia maxim.sidorov@uni-ulm.de, alexander.schmitt@uni-ulm.de, eugene.semenkin@sibsau.ru, wolfgang.minker@uni-ulm.de Abstract Emotion Recognition (ER) is an important part of dialogue analysis which can be used in order to improve the quality of Spoken Dialogue Systems (SDSs). The emotional hypothesis of the current response of an end-user might be utilised by the dialogue manager component in order to change the SDS strategy which could result in a quality enhancement. In this study additional speaker-related information is used to improve the performance of the speech-based ER process. The analysed information is the speaker identity, gender and age of a user. Two schemes are described here, namely, using additional information as an independent variable within the feature vector and creating separate emotional models for each speaker, gender or age-cluster independently. The performances of the proposed approaches were compared against the baseline ER system, where no additional information has been used, on a number of emotional speech corpora of German, English, Japanese and Russian. The study revealed that for some of the corpora the proposed approach significantly outperforms the baseline methods with a relative difference of up to 11.9%. Keywords: Adaptive speech-based emotion recognition, classification performances, corpora evaluation. 1. Introduction By deploying the ER component within the SDS, its qual- ity could be significantly increased. It might be benefi- cial during human-robot or even human-human interaction. Whereas the majority of studies concentrate on speaker- independent ER experiments, in some cases speaker- awareness can bring an additional advantage. Despite the fact that the basic emotions are shared between cultures and nationalities (Scherer, 2002) obviously, each person ex- presses his emotions individually. This thesis lies behind the idea of building different emotional models for each speaker independently or incorporating the speaker-specific information within the single ER model in a different way. On the one hand it results in a problem-decomposition, sim- ilar to the cluster-then-classify approach, but on the other hand by deploying different models for each speaker, the individual features of the corresponding speaker can be caught and utilised properly. Furthermore, as has been mentioned in many studies (Brody, 1985), (Hall et al., 2000) the gender difference in emotional expression has been detected during several psy- chological investigations. In contrast to the very specific nature of speaker-adaptive ER, gender-adaptive ER might be more general. A similar idea is behind the age-adaptive ER models, where each user has one of the age-specific la- bels (for example youth or adult). The global aim of the study is to figure out whether the speaker-, gender- or even age-related information of an end- user might be utilised in order to improve the quality of the ER models. We proposed here a two-stage approach, where firstly the speaker or other additional information (gender or age) is identified and secondly, an adaptive ER proce- dure is performed. We intend to study both cases: the the- oretically …

Content truncated to 500 words