Generating a Gold Standard for a Swedish Sentiment Lexicon

Generating a Gold Standard for a Swedish Sentiment Lexicon Jacobo Rouces, Nina Tahmasebi, Lars Borin, Stian Rødven Eide Sprakbanken, University of Gothenburg {jacobo.rouces, nina.tahmasebi, lars.borin, stian.rodven.eide}@gu.se Abstract There is an increasing demand for multilingual sentiment analysis, and most work on sentiment lexicons is still carried out based on English lexicons like WordNet. In addition, many of the non-English sentiment lexicons that do exist have been compiled by (machine) translation from English resources, thereby arguably obscuring possible language-specific characteristics of sentiment-loaded vocabulary. In this paper we describe the creation of a gold standard for the sentiment annotation of Swedish terms as a first step towards the creation of a full- fledged sentiment lexicon for Swedish – i.e., a lexicon containing information about prior sentiment (also called polarity) values of lexical items (words or disambiguated word senses), along a scale negative–positive. We create a gold standard for sentiment annotation of Swedish terms, using the freely available SALDO lexicon and the Gigaword corpus. For this purpose, we employ a multi-stage approach combining corpus-based frequency sampling and two stages of human annotation: direct score annotation followed by Best-Worst Scaling. In addition to obtaining a gold standard, we analyze the data from our process and we draw conclusions about the optimal sentiment model. Keywords: sentiment analysis, Swedish, gold standard, lexical resource 1. Introduction There is an increasing demand for multilingual sentiment analysis, and most work on sentiment lexicons is still car- ried out based on English lexicons like WordNet (Fellbaum, 1998). In addition, many of the non-English sentiment lex- icons that do exist have been compiled by (machine) trans- lation from English resources, e.g., by Mohammad and Turney (2010)1 and Chen and Skiena (2014), thereby ar- guably obscuring possible language-specific characteristics of sentiment-loaded vocabulary. In this paper we describe the creation of a gold standard for the sentiment annotation of Swedish terms as a first step towards the creation of a full- fledged sentiment lex- icon for Swedish – i.e., a lexicon containing information about prior sentiment (also called polarity) values of lex- ical items (words or disambiguated word senses), along a scale negative–positive. For this purpose, we use human annotations of items sampled from a general-purpose com- putational lexical resource. More specifically, we employ a multi-stage approach combining corpus-based frequency sampling, direct score annotation and Best- Worst Scaling (BWS) (Kiritchenko and Mohammad, 2016). 2. State of the art We base our gold standard on SALDO (Språkbanken, 2015a), which is an existing Swedish lexical-semantic com- putational resource (Borin et al., 2013). It is organized as a lexical-semantic network of word senses, whose topol- ogy reflects semantic distance among the word senses. Each word sense in SALDO is additionally connected to one or more form units (lemmas plus part of speech and full inflectional and compounding information). These are formally organized as an independent lexical resource – SALDO’s Morphology (Språkbanken, 2015b) – which con- sequently can be used in NLP applications independently of 1http://saifmohammad.com/WebPages/ NRC-Emotion-Lexicon.htm SALDO, e.g., for lemmatization and morphological anal- ysis of Swedish text. …

Content truncated to 500 words