A Fine-grained Sentiment Dataset for Norwegian

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5025–5033 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5025 A Fine-Grained Sentiment Dataset for Norwegian Lilja Øvrelid, Petter Mæhlum, Jeremy Barnes, Erik Velldal University of Oslo Department of Informatics {liljao,pettemae,jeremycb,erikve}@ifi.uio.no Abstract We introduce NoReCfine, a dataset for fine-grained sentiment analysis in Norwegian, annotated with respect to polar expressions, targets and holders of opinion. The underlying texts are taken from a corpus of professionally authored reviews from multiple news-sources and across a wide variety of domains, including literature, games, music, products, movies and more. We here present a detailed description of this annotation effort. We provide an overview of the developed annotation guidelines, illustrated with examples, and present an analysis of inter-annotator agreement. We also report the first experimental results on the dataset, intended as a preliminary benchmark for further experiments. Keywords: Sentiment analysis, opinion mining, Norwegian 1. Introduction In this work, we describe the annotation of a fine-grained sentiment dataset for Norwegian, analysing opinions in terms of their polar expressions, targets, and holders. The dataset, including the annotation guidelines, is made pub- licly available1 and is the first of its kind for Norwegian. The underlying texts are taken from the Norwegian Review Corpus (NoReC) (Velldal et al., 2018) – a corpus of profes- sionally authored reviews across a wide variety of domains, including literature, video games, music, various product categories, movies, TV-series, restaurants, etc. In Mæhlum et al. (2019), a subset of the documents, dubbed NoReCeval, were annotated at the sentence-level, indicating whether a sentence contains an evaluation or not. These prior anno- tations strictly indicated evaluativeness and did not include negative or positive polarity, as this can be mixed at the sentence-level. In the current work, the previous annotation effort has been considerably extended to include the span of polar expressions and the corresponding targets and hold- ers of the opinion. We also indicate the intensity of the pos- itive or negative polarity on a three-point scale, along with a number of other attributes of the expressions. The result- ing dataset, dubbed NoReCfine, comprises almost 8000 sen- tences (of which roughly half are evaluative) across roughly 300 reviews and includes both subjective and fact-implied evaluations. In addition to discussing annotation principles and examples, we also present the first experimental results. The paper is structured as follows. Section 2 reviews re- lated work, both in terms of related resources for other languages and work on computational modeling of fine- grained opinions. We then go on to discuss our annota- tion effort in Section 3, where we describe annotation prin- ciples, discuss a number of examples and finally present statistics on inter-annotator agreement. Section 5 presents our first experiments using this dataset for neural machine learning of fine-grained opinions, before Section 6 dis- cusses some future directions of research. Finally, Section 7 summarizes the main contributions of the paper. 1https://github.com/ltgoslo/norec_fine 2. Related Work Fine-grained approaches to sentiment analysis include opinion annotations as …

Content truncated to 500 words