AttnConvnet at SemEval-2018 Task 1: Attention-based Convolutional Neural Networks for Multi-label Emotion Classification

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 141–145 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics AttnConvnet at SemEval-2018 Task 1: Attention-based Convolutional Neural Networks for Multi-label Emotion Classification Yanghoon Kim1,2, Hwanhee Lee1 and Kyomin Jung1,2 1Seoul National University, Seoul, Korea 2Automation and Systems Research Institute, Seoul National University, Seoul, Korea {ad26kr,wanted1007,kjung}@snu.ac.kr Abstract In this paper, we propose an attention-based classifier that predicts multiple emotions of a given sentence. Our model imitates human’s two-step procedure of sentence understanding and it can effectively represent and classify sentences. With emoji-to-meaning preprocess- ing and extra lexicon utilization, we further improve the model performance. We train and evaluate our model with data provided by SemEval-2018 task 1-5, each sentence of which has several labels among 11 given emo- tions. Our model achieves 5th/1st rank in En- glish/Spanish respectively. 1 Introduction Since the revolution in deep neural networks, es- pecially with the help of Long short-term mem- ory(Hochreiter and Schmidhuber, 1997), it has been easy for machines to imitate human’s linguis- tic activities, such as sentence classification(Kim, 2014), language model(Sundermeyer et al., 2010), machine translation(Bahdanau et al., 2015). Emotion classification is a subpart of sentence classification that predicts the emotion of the given sentence by understanding the meaning of it. Multi-label emotion classification requires more powerful ability to comprehend the sentence in variety of aspects. For example, given a sen- tence ’For real? Look what I got for my birth- day present!!’, it is easy for human to figure out that the sentence not only expressing ’joy’ but also ’surprise’. However, machines may require more task-specific structure to solve the same problem. Attention mechanisms are one of the most spot- lighted trends in deep learning and recently made their way into NLP. Applied to systems with neu- ral networks, it functions as visual attention mech- anisms found in humans(Denil et al., 2012) and the most effective region of features will be high- lighted over time, making the system better exploit the features related to the training objective. (Bah- danau et al., 2015) is one of the most significant footprints of attention mechanism in NLP and they applied attention mechanisms to machine transla- tion for the first time. The model generates target word under the influence of related source words. Furthermore, Vaswani et al. (2017) proposed a brand new architecture for neural machine transla- tion. The model utilizes attention mechanisms not only as the submodule but also as the main struc- ture, improving time complexity and performance. Inspired by (Vaswani et al., 2017), we come up with attention-based multi-label sentence clas- sifier that can effectively represent and classify sentences. Our system is composed of a self- attention module and multiple CNNs enabling it to imitate human’s two-step procedure of analyz- ing sentences: comprehend and classify. Further- more, our emoji-to-meaning preprocessing and extra lexicon utilization improve model perfor- mance on given dataset. We evaluated our system on the dataset of (Mohammad et al., 2018), where it ranked 5th/1st rank in …

Content truncated to 500 words