NTUA-SLP at SemEval-2018 Task 1: Predicting Affective Content in Tweets with Deep Attentive RNNs and Transfer Learning

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 245–255 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics NTUA-SLP at SemEval-2018 Task 1: Predicting Affective Content in Tweets with Deep Attentive RNNs and Transfer Learning Christos Baziotis1,3, Nikos Athanasiou1, Alexandra Chronopoulou1, Athanasia Kolovou1,2, Georgios Paraskevopoulos1,4, Nikolaos Ellinas1 Shrikanth Narayanan4,5, Alexandros Potamianos1,4,5 1School of ECE, National Technical University of Athens, Athens, Greece 2 Department of Informatics, University of Athens, Athens, Greece 3 Department of Informatics, Athens University of Economics and Business, Athens, Greece 4 Behavioral Signal Technologies, Los Angeles, CA 5 Signal Analysis and Interpretation Laboratory (SAIL), USC, Los Angeles, USA cbaziotis@mail.ntua.gr, el12074@central.ntua.gr el12068@central.ntua.gr, akolovou@di.uoa.gr geopar@central.ntua.gr, nellinas@central.ntua.gr shri@sipi.usc.edu, potam@central.ntua.gr Abstract In this paper we present deep-learning mod- els that submitted to the SemEval-2018 Task 1 competition: “Affect in Tweets”. We par- ticipated in all subtasks for English tweets. We propose a Bi-LSTM architecture equipped with a multi-layer self attention mechanism. The attention mechanism improves the model performance and allows us to identify salient words in tweets, as well as gain insight into the models making them more interpretable. Our model utilizes a set of word2vec word em- beddings trained on a large collection of 550 million Twitter messages, augmented by a set of word affective features. Due to the limited amount of task-specific training data, we opted for a transfer learning approach by pretrain- ing the Bi-LSTMs on the dataset of Semeval 2017, Task 4A. The proposed approach ranked 1st in Subtask E “Multi-Label Emotion Classi- fication”, 2nd in Subtask A “Emotion Intensity Regression” and achieved competitive results in other subtasks. 1 Introduction Social media content has dominated online com- munication, enriching and changing language with new syntactic and semantic constructs that allow users to express facts, opinions and emotions in short amount of text. The analysis of such con- tent has received great attention in NLP research due to the wide availability of data and the inter- esting language novelties. Specifically the study of affective content in Twitter has resulted in a va- riety of novel applications, such as tracking prod- uct perception (Chamlertwat et al., 2012), public opinion detection about political tendencies (Pla <user> has forever changed my life <hashtag> blessed </hashtag> Emotions: joy, love, optimism seriously about to smack someone in the face <hashtag> arsehole </hashtag> Emotions: anger, disgust Figure 1: Attention heat-map visualization. The color intensity corresponds to the weight given to each word by the self-attention mechanism. and Hurtado, 2014; Tumasjan et al., 2010), stock market monitoring (Si et al., 2013; Bollen et al., 2011b) etc. The wide usage of figurative language, such as emojis and special language forms like ab- breviations, hashtags, slang and other social me- dia markers, which do not align with the conven- tional language structure, make natural language processing in Twitter even more challenging. In the past, sentiment analysis was tackled by extracting hand-crafted features or features from sentiment lexicons (Nielsen, 2011; Mohammad and Turney, 2010, 2013; Go et al., 2009) that were fed to classifiers …

Content truncated to 500 words