Lexicon-based Sentiment Analysis in German: Systematic Evaluation of Resources and Preprocessing Techniques

Lexicon-based Sentiment Analysis in German: Systematic Evaluation of Resources and Preprocessing Techniques Jakob Fehle Media Informatics Group University of Regensburg Regensburg, Germany jakob.fehle@ur.de Thomas Schmidt Media Informatics Group University of Regensburg Regensburg, Germany thomas.schmidt@ur.de Christian Wolff Media Informatics Group University of Regensburg Regensburg, Germany christian.wolff@ur.de Abstract We present the results of an evaluation study in the context of lexicon-based sentiment analy- sis resources for German texts. We have set up a comprehensive compilation of 19 sentiment lexicon resources and 20 sentiment-annotated corpora available for German across multiple domains. In addition to the evaluation of the sentiment lexicons we also investigate the in- fluence of the following preprocessing steps and modifiers: stemming and lemmatization, part-of-speech-tagging, usage of emoticons, stop words removal, usage of valence shifters, intensifiers, and diminishers. We report the best performing lexicons as well as the influ- ence of preprocessing steps and other modifi- cations on average performance across all cor- pora. We show that larger lexicons with contin- uous values like SentiWS and SentiMerge per- form best across the domains. The best per- forming configuration of lexicon and modifi- cations considering the f1-value and accuracy averages across all corpora achieves around 67%. Preprocessing, especially stemming or lemmatization increases the performance con- sistently on average around 6% and for certain lexicons and configurations up to 16.5% while methods like the usage of valence shifters, in- tensifiers or diminishers rarely influence over- all performance. We discuss domain-specific differences and give recommendations for the selection of lexicons, preprocessing and modi- fications. 1 Introduction Sentiment analysis (also often referred to as opin- ion mining) is a sub-field of affective computing, which deals with the detection and analysis of hu- man sentiment and emotions in various application areas like game design (Halbhuber et al., 2019), health (Hartl et al., 2019) and human-computer interaction (Ortloff et al., 2019). Sentiment anal- ysis focuses on text as modality and refers to the task of classifying texts of various lengths con- cerning polarity (or valence) expressed in the text, meaning whether the sentiment of a text is rather positive or negative (Liu, 2015). Application ar- eas for sentiment analysis in natural language pro- cessing (NLP) are social media content (M¨antyl¨a et al., 2018), social sciences (Schmidt et al., 2020b), health (Moßburger et al., 2020), user-generated content (Schmidt et al., 2020a) , digital humanities (Kim and Klinger, 2018a), and human-computer interaction (Schmidt et al., 2020c) to name just a few examples. Methods for performing sentiment analysis can be divided into two major branches: lexicon-based (also often referred to as rule-based or dictionary- based methods; Taboada et al., 2011) and ma- chine learning (ML)-based approaches. Lexicon- based sentiment analysis uses lexicons consisting of words that are pre-annotated concerning their sentiment expression, which we refer to as senti- ment bearing words (SBWs). There are multiple ways to create and acquire such lexicons like crowd- sourcing, expert annotations or semi-automatic ap- proaches (cf. Ribeiro et al., 2016). Values of SBWs can either be binary, e.g. +1 (positive) and -1 (neg- ative) …

Content truncated to 500 words