Semi-Automatic Construction and Refinement of an Annotated Corpus for a Deep Learning Framework for Emotion Classification

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1611–1617 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1611 Semi-Automatic Construction and Refinement of an Annotated Corpus for a Deep Learning Framework for Emotion Classification Jiajun Xu1,2, Kyosuke Masuda2, Hiromitsu Nishizaki2, Fumiyo Fukumoto2, Yoshimi Suzuki2 1Hangzhou Dianzi University, Hangzhou, Zhejiang, 310018, China 2University of Yamanashi, 4-3-11 Takeda, Kofu-shi, Yamanashi 400-8511 Japan {jiajunxu,kyosuke}@alps-lab.org, {hnishi, fukumoto, ysuzuki}@yamanashi.ac.jp Abstract In the case of using a deep learning (machine learning) framework for emotion classification, one significant difficulty faced is the requirement of building a large, emotion corpus in which each sentence is assigned emotion labels. As a result, there is a high cost in terms of time and money associated with the construction of such a corpus. Therefore, this paper proposes a method of creating a semi-automatically constructed emotion corpus. For the purpose of this study sentences were mined from Twitter using some emotional seed words that were selected from a dictionary in which the emotion words were well-defined. Tweets were retrieved by one emotional seed word, and the retrieved sentences were assigned emotion labels based on the emotion category of the seed word. It was evident from the findings that the deep learning-based emotion classification model could not achieve high levels of accuracy in emotion classification because the semi-automatically constructed corpus had many errors when assigning emotion labels. In this paper, therefore, an approach for improving the quality of the emotion labels by automatically correcting the errors of emotion labels is proposed and tested. The experimental results showed that the proposed method worked well, and the classification accuracy rate was improved to 55.1% from 44.9% on the Twitter emotion classification task. Keywords: emotion classification, construction of emotion corpus, deep learning, dataset refinement, Twitter 1. Introduction Recently, various cognitive problems, such as image (Deng et al., 2009) and speech recognition (Hinton et al., 2012), have been solved by deep learning-based machine learn- ing frameworks. A deep learning-based cognitive system needs a large number of datasets for model training to pre- vent the model from over-training (over-fitting). A dataset for model training usually consists of a pair of input data and its annotated teacher label. However, the construc- tion of a dataset is highly costly because in the past hu- mans have usually had to annotate the teacher labels to the dataset. This has resulted in more cost-effective pro- posals of data augmentation methods (Cubuk et al., 2019; Nishizaki, 2017; Kafle et al., 2017) being put forward. Al- though it is comparatively easy to perform data augmenta- tion for image and audio datasets, data augmentation for a text corpus for emotion classification is considerably more difficult. This paper describes an emotion classification of text sen- tences from Twitter 1, one of the most popular social net- working services. To deal with an emotion classification task, a training dataset has to be prepared and constructed for the emotion classifiers. For example, SemEval-2018 Task 1 (Affect in Tweets)(Mohammad …

Content truncated to 500 words