A Japanese Dataset for Subjective and Objective Sentiment Polarity Classification in Micro Blog Domain

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7022–7028 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7022 A Japanese Dataset for Subjective and Objective Sentiment Polarity Classification in Micro Blog Domain Haruya Suzuki†, Yuto Miyauchi†, Kazuki Akiyama†, Tomoyuki Kajiwara†, Takashi Ninomiya†, Noriko Takemura‡, Yuta Nakashima‡, Hajime Nagahara‡ †Ehime University, ‡Osaka University {suzuki, miyauchi, k akiyama}@ai.cs.ehime-u.ac.jp {kajiwara, ninomiya}@cs.ehime-u.ac.jp {takemura, n-yuta, nagahara}@ids.osaka-u.ac.jp Abstract We annotate 35,000 SNS posts with both the writer’s subjective sentiment polarity labels and the reader’s objective ones to construct a Japanese sentiment analysis dataset. Our dataset includes intensity labels (none, weak, medium, and strong) for each of the eight basic emotions by Plutchik (joy, sadness, anticipation, surprise, anger, fear, disgust, and trust) as well as sentiment polarity labels (strong positive, positive, neutral, negative, and strong negative). Previous studies on emotion analysis have studied the analysis of basic emotions and sentiment polarity independently. In other words, there are few corpora that are annotated with both basic emotions and sentiment polarity. Our dataset is the first large-scale corpus to annotate both of these emotion labels, and from both the writer’s and reader’s perspectives. In this paper, we analyze the relationship between basic emotion intensity and sentiment polarity on our dataset and report the results of benchmarking sentiment polarity classification. Keywords: sentiment analysis, emotion detection, Japanese 1. Introduction Emotion analysis is one of the major natural language processing tasks with many applications such as dia- logue systems (Tokuhisa et al., 2008) and social me- dia mining (Stieglitz and Dang-Xuan, 2013). Previous studies in emotion analysis have addressed two sub- tasks: classification of sentiment polarity (Sentiment Analysis) and classification of basic emotions (Emo- tion Detection). The former targets positive or negative sentiment polarity, while the latter targets about four to eight basic emotions, such as joy and sadness (Plutchik, 1980; Ekman, 1992). As shown in Table 1, previous studies on emotion anal- ysis have been diverse in terms of whether it deals with sentiment polarity (Sentiment) or basic emotions (Emotion), and whether it deals with the emotions of the writer of the text (Subjective Emotion) or the emo- tions of the reader of the text (Objective Emotion). However, the relationship between sentiment polarity and basic emotions, as well as the relationship between the writer’s emotions and the reader’s ones, are not clear, as there is no previous study that has compre- hensively addressed these issues. In this study, to extend the WRIME dataset (Kajiwara et al., 2021) with basic emotion intensity from both subjective and objective standpoints for Japanese SNS text, we annotated sentiment polarity labels from both subjective and objective standpoints for these texts. Specifically, one subjective annotator and three objec- tive annotators annotated both labels of the sentiment polarity and the intensity of each of Plutchik’s eight emotions (joy, sadness, anticipation, surprise, anger, fear, disgust, and trust) to 35,000 SNS posts collected from 60 subjective annotators. We annotated emotional intensity with a four-point scale (none, weak, medium, and strong) …

Content truncated to 500 words