Multi-Level Sentiment Analysis of PolEmo 2.0: Extended Corpus of Multi-Domain Consumer Reviews

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 980–991 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 980 Multi-Level Sentiment Analysis of PolEmo 2.0: Extended Corpus of Multi-Domain Consumer Reviews Jan Koco´n Wrocław University of Science and Technology Wrocław, Poland jan.kocon @pwr.edu.pl Piotr Miłkowski Wrocław University of Science and Technology Wrocław, Poland piotr.milkowski @pwr.edu.pl Monika Za´sko-Zieli´nska University of Wrocław Institute of Polish Studies Wrocław, Poland monika.zasko-zielinska @uwr.edu.pl Abstract In this article we present an extended version of PolEmo – a corpus of consumer reviews from 4 domains: medicine, hotels, products and school. Current version (PolEmo 2.0) con- tains 8,216 reviews having 57,466 sentences. Each text and sentence was manually anno- tated with sentiment in 2+1 scheme, which gives a total of 197,046 annotations. We ob- tained a high value of Positive Specific Agree- ment, which is 0.91 for texts and 0.88 for sentences. PolEmo 2.0 is publicly available under a Creative Commons copyright license. We explored recent deep learning approaches for the recognition of sentiment, such as Bi- directional Long Short-Term Memory (BiL- STM) and Bidirectional Encoder Representa- tions from Transformers (BERT). 1 Introduction In recent years, we have observed a growing in- terest in methods of effective sentiment analysis, especially in subjective, opinion-forming online texts. This trend is perfectly illustrated by Fig- ure 1, which compares the popularity of two terms: customer feedback and sentiment analysis. A very dynamic growth has been observed since 2010, which correlates with the increase in the number of scientific research in this area. Many stud- ies focus on the perception of emotion and sen- timent in text messages and, for example, their impact on election results (Ramteke et al., 2016), prediction of future events (Zhang and Skiena, 2010) and security issues around the world (Sub- ramaniyaswamy et al., 2017; Al-Rowaily et al., 2015). Automatic sentiment analysis systems have proven to be effective in analyzing many differ- ent types of text data such as emails, blogs, news, tweets and books (Medhat et al., 2014). The in- troduction of advanced computational techniques (machine learning, deep learning) in natural lan- 2004-01 2004-09 2005-05 2006-01 2006-09 2007-05 2008-01 2008-09 2009-05 2010-01 2010-09 2011-05 2012-01 2012-09 2013-05 2014-01 2014-09 2015-05 2016-01 2016-09 2017-05 2018-01 2018-09 2019-05 0 10 20 30 40 50 60 70 80 90 100 customer feedback sentiment analysis Figure 1: Google Trends (https://trends.google. com) data showing interest in time for search terms "cus- tomer feedback" and "sentiment analysis". On the vertical axis 100 means biggest search term popularity. guage processing has resulted in a significant in- crease in sentiment analysis techniques (Zhang et al., 2018). This increase for some languages is effectively limited by the lack of good quality resources for this task, especially in the form of manually annotated corpora (Balahur and Turchi, 2012; Dashtipour et al., 2016). Analysis of the existing language resources in the area of sentiment analysis shows that they largely concern the English language (Dashtipour et al., 2016). However, there is a clear grow- …

Content truncated to 500 words