DMCB at SemEval-2018 Task 1: Transfer Learning of Sentiment Classification Using Group LSTM for Emotion Intensity prediction

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 300–304 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics DMCB at SemEval-2018 Task 1: Transfer Learning of Sentiment Classification Using Group LSTM for Emotion Intensity prediction Youngmin Kim, Hyunju Lee Gwangju Institute of Science and Technology, Data mining and Computational Biology Lab, Gwangju, Korea {minok00, hyunjulee}@gist.ac.kr Abstract This paper describes a system attended in the SemEval-2018 Task 1 “Affect in tweets” that predicts emotional intensities. We use Group LSTM with an attention model and transfer learning with sentiment classification data as a source data (SemEval 2017 Task 4a). A trans- fer model structure consists of a source domain and a target domain. Additionally, we try a new dropout that is applied to LSTMs in the Group LSTM. Our system ranked 8th at the subtask 1a (emotion intensity regression). We also show various results with different archi- tectures in the source, target and transfer mod- els. 1 Introduction Sentiment analysis is one of the most famous Nat- ural Language Process (NLP) task. In this study, we perform a task that predicts emotional inten- sities of anger, joy, fear and sadness with tweet messages, where intensity values range from 0 to 1. This task is competed at SemEval-2018 Task 1 (Mohammad et al., 2018). In previous stud- ies, neural networks with word embedding and affective lexicons were widely used (Goel et al., 2017; He et al., 2017). Also, many studies em- ployed support vector regression (Duppada and Hiray, 2017; Akhtar et al., 2017). Transfer learning was recently proposed as an effecive approach to have higher performance, when data is not abundant. Using a pre-trained deep-learning model with an abundant data set has been popular and shows good results in various tasks (Donahue et al., 2014; Conneau et al., 2017). Especially in a medical image task, it is very effi- cient because of lacks of medical data (Tajbakhsh et al., 2016). Just as humans can learn new things better with their past knowledge, neural networks can also be trained on target domains by transfer- ring knowledge from the source domain. We make a transfer model that can be divided into a source model and a target model. The source model is constructed based on the paper (Baziotis et al., 2017). The model of this paper uses LSTM with attention. However, we introduce Group LSTM (GLSTM) (Kuchaiev and Ginsburg, 2017) with a new dropout. After then, we make the target model with LSTM. In the result section, we provide comparison of LSTM and GLSTM in the source model, and re- sults of various pre-trained word embeddings with target model. Finally, we discuss about the result of the transfer model that is a combined model with the source and target models. 2 System Description 2.1 Data and Label For transfer learning, we use a source data pro- vided by SemEval 2017 Task4 (a) (Rosenthal et al., 2017). The task of the source domain is to classify sentences to positive, negative and …

Content truncated to 500 words