Towards Speech-only Opinion-level Sentiment Analysis

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2000–2006 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2000 Towards Speech-only Opinion-level Sentiment Analysis Annalena Aicher1, Alisa Vinogradova2, Aleksey Gusev2, Yury Matveev2, Wolfgang Minker1 1 Ulm University, Albert-Einstein-Allee 43, 89075 Ulm 2 Kronverksky Pr. 49, bldg. A, St. Petersburg, 197101, Russia annlena.aicher@uni-ulm.de Abstract The growing popularity of various forms of Spoken Dialogue Systems (SDS) raises the demand for their capability of implicitly assessing the speaker’s sentiment from speech only. Mapping the latter on user preferences enables to adapt to the user and individualize the requested information to increase user satisfaction. In this paper, we explore the integration of rank consistent ordinal regression into a speech-only sentiment prediction task performed by ResNet-like systems. Furthermore, we use speaker verification extractors trained on larger datasets as low-level feature extractors. An improvement of performance is shown by fusing sentiment and pre-extracted speaker embeddings reducing the speaker bias of sentiment predictions. Numerous experiments on CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) databases show that we beat the baselines of state-of-the-art unimodal approaches. Using speech as the only modality combined with optimizing an order-sensitive objective function gets significantly closer to the sentiment analysis results of state-of-the-art multimodal systems. Keywords: speech sentiment analysis, sentiment intensity prediction, (speech) emotion recognition, acoustic emotion recognition, CMU-MOSEI, COnsistent RAnk Logits (CORAL), spoken dialogue systems, HCI, computational paralinguistics 1. Introduction In the past decade, the development of spoken dialogue systems and the variety of techniques to meet the users’ needs have grown rapidly. For instance, natural lan- guage displays a straightforward way to communicate for humans and thus, also interactions with comput- ers via spoken language are perceived more comfort- able and natural for users. Therefore, it is required that SDS are capable of managing complex, human- like interactions and learn how to adapt to users and their preferences. There are various approaches to de- tect user preferences in SDS. Many literature refer- ences estimate user preferences only in a static manner e.g. through historical user behaviors and profiles or by exploiting the interaction history (Liu and Mazumder, 2021). In Gao et al. (2021) conversational recom- mender systems and their possibilities to explicitly ob- tain the exact preference of users are investigated. An- other explicit approach is shown by Zeng et al. (2018) where the user’s preferences on food are elicited by in- terviewing the user directly. Furthermore, we (Aicher et al., 2021) introduced the argumentative dialogue sys- tem BEA which estimates the users’ preferences (pre- ferring or rejecting a presented argument) via explicit user feedback by using weighted Bipolar Argumenta- tion Graphs (wBAGs) (Amgoud and Ben-Naim, 2016; Amgoud and Ben-Naim, 2018). As explicit feedback entails the risk of a repetitive course of the dialogue, which might bore or annoy the user, instead we aim for an implicit preference detection. Rach et al. (2019) previously suggested to use the information on multi- modal emotion recognition techniques to implicitly ob- tain user preferences …

Content truncated to 500 words