How to Estimate Continuous Sentiments From Texts Using Binary Training Data

How to Estimate Continuous Sentiments From Texts Using Binary Training Data Sandra Wankm¨uller Ludwig-Maximilians-Universit¨at Munich, Germany https://orcid.org/0000-0002-4003-1704 sandra.wankmueller@gsi.lmu.de Christian Heumann Ludwig-Maximilians-Universit¨at Munich, Germany chris@stat.uni-muenchen.de Abstract Although sentiment is conceptualized as a con- tinuous variable, most text-based sentiment analyses categorize texts into discrete senti- ment categories. Compared to discrete catego- rizations, continuous sentiment estimates pro- vide much more detailed information which can be used for more fine-grained analyses by researchers and practitioners alike. Yet, exist- ing approaches that estimate continuous senti- ments either require detailed knowledge about context and compositionality effects or require granular training labels, that are created in re- source intensive annotation processes. Thus, existing approaches are too costly to be ap- plied for each potentially interesting applica- tion. To overcome this problem, this work in- troduces CBMM (standing for classifier-based beta mixed modeling procedure). CBMM ag- gregates the predicted probabilities of an en- semble of binary classifiers via a beta mixed model and thereby generates continuous, real- valued output based on mere binary training in- put. CBMM is evaluated on the Stanford Senti- ment Treebank (SST) (Socher et al., 2013), the V-reg data set (Mohammad et al., 2018), and data from the 2008 American National Elec- tion Studies (ANES) (The American National Election Studies, 2015). The results show that CBMM produces continuous sentiment esti- mates that are acceptably close to the truth and not far from what could be obtained if highly fine-grained training data were available. 1 Introduction In natural language processing and computer sci- ence, the term sentiment typically refers to a loosely defined, broad umbrella concept: Feeling, emotion, judgement, evaluation, and opinion all fall under the term sentiment or are used synonymously with it (Pang and Lee, 2008; Liu, 2015). Interestingly, the broad notion of sentiment is very well cap- tured by the psychological concept of an attitude (Liu, 2015). In psychology, scholars agree that an attitude is a summary evaluation of an entity (Ba- naji and Heiphetz, 2010; Albarracin et al., 2019). An attitude is the aggregated evaluative response resulting from a multitude of different (and po- tentially conflicting) information bases relating to the attitude entity (Fabringar et al., 2019). When putting the definition of an attitude as an evaluative summary into mathematical terms, an attitude is a unidimensional, continuous variable ranging from highly negative to highly positive (Cacioppo et al., 1997). This notion that attitudes are continuous is also mirrored in the sentiment analysis literature in which sentiments are devised to vary in their levels of intensity (Liu, 2015). Despite this conceptualization, in an overwhelm- ing majority of studies textual sentiment expres- sions are measured as instances of discrete classes. Sentiment analysis often implies a binary or multi- class classification task in which texts are assigned into two or three classes, thereby distinguishing positive from negative sentiments and sometimes a third neutral category (e.g. Pang et al., 2002; Tur- ney, 2002; Maas et al., 2011). Other studies pursue ordinal sentiment classification (e.g. Pang and Lee, 2005; Thelwall et al., 2010; Socher et al., …

Content truncated to 500 words