Training a Broad-Coverage German Sentiment Classification Model for Dialog Systems

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1627–1632 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1627 Training a Broad-Coverage German Sentiment Classification Model for Dialog Systems Oliver Guhr, Anne-Kathrin Schumann, Frank Bahrmann, Hans-Joachim B¨ohme University of Applied Science (HTW) Dresden, Germany T2K: Text to Knowledge, Dresden, Germany {oliver.guhr, frank.bahrmann, hans-joachim.boehme}@htw-dresden.de anne-kathrin.schumann@text2knowledge.de Abstract This paper describes the training of a general-purpose German sentiment classification model. Sentiment classification is an important aspect of general text analytics. Furthermore, it plays a vital role in dialogue systems and voice interfaces that depend on the ability of the system to pick up and understand emotional signals from user utterances. The presented study outlines how we have collected a new German sentiment corpus and then combined this corpus with existing resources to train a broad-coverage German sentiment model. The resulting data set contains 5.4 million labelled samples. We have used the data to train both, a simple convolutional and a transformer-based classification model and compared the results achieved on various training configurations. The model and the data set will be published along with this paper. Keywords: sentiment analysis, German sentiment model, German corpus 1. Introduction The presented work is carried out within the context of developing service robots for public spaces and nursing homes. The robots are deployed in different scenarios, for instance as tour guides in museums and as assistants in health care settings. In both situations, it is crucial that the robot, or more precisely, its voice user interface, is able to adapt to users’ needs and intelligently respond to users’ emotions. We believe that the ability to pick up and un- derstand emotional signals provided by the users’ language input is crucial not only for the acceptance of service robots but also for the handling of situations where the users pro- vide strong emotional feedback. There are two main applications for sentiment analysis in the context of a robots’ conversational voice interface: • We want to classify the sentiment of all user state- ments. This will enable the dialogue manager of the robot to respond to positive and negative user feed- back. • Moreover, the users’ utterances and their sentiment la- bels can later be used to rate conversations and evalu- ate the users’ reactions to changes in the behaviour of the robot. This concept is common and other conver- sational voice interfaces like (Chen et al., 2018) and (Fang et al., 2018) also apply sentiment analysis for these tasks. The development of reliable machine learning models for the mentioned tasks requires annotated training data. How- ever, available German sentiment data sets such as PotTS (Sidarenka, 2016), SB10k (Cieliebak et al., 2017), and GermEval-2017 (Wojatzki et al., 2017), when combined, comprise only 39,000 sentences. In contrast, commonly used English data sets such as IMDB (Maas et al., 2011) contain 50,000 binary labeled items and the YELP data set (Zhang et al., 2016) contains 598,000 binary labeled items. Another aspect that is …

Content truncated to 500 words