psyML at SemEval-2018 Task 1: Transfer Learning for Sentiment and Emotion Analysis

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 369–376 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics psyML at SemEval-2018 Task 1: Transfer Learning for Sentiment and Emotion Analysis Grace Gee, Eugene Wang psyML, Inc. grace.eugene@psyml.co Abstract In this paper, we describe the first attempt to perform transfer learning from sentiment to emotions. Our system employs Long Short- Term Memory (LSTM) networks, including bidirectional LSTM (biLSTM) and LSTM with attention mechanism. We perform trans- fer learning by first pre-training the LSTM net- works on sentiment data before concatenating the penultimate layers of these networks into a single vector as input to new dense layers. For the E-c subtask, we utilize a novel approach to train models for correlated emotion classes. Our system performs 4/48, 3/39, 8/38, 4/37, 4/35 on all English subtasks EI-reg, EI-oc, V- reg, V-oc, E-c of SemEval 2018 Task 1: Affect in Tweets. 1 Introduction SemEval-2018 Task 1: Affect in Tweets (Moham- mad et al., 2018) is a shared task expanding on pre- vious SemEval sentiment tasks and the WASSA- 2017 Shared Task on Emotion Intensity (Moham- mad and Bravo-Marquez, 2017). It presents 5 tasks: 1. Emotion intensity regression (EI-reg): given tweet and emotion (fear, anger, joy or sad- ness), predict real-valued emotion intensity from 0 to 1. 2. Emotion intensity ordinal classification (EI- oc): given tweet and emotion (fear, anger, joy or sadness), predict emotion intensity ordinal class from 0 (no emotion) to 3 (high emo- tion). 3. Sentiment intensity regression (V-reg): given tweet, predict real-valued sentiment intensity from 0 (no sentiment) to 1 (high sentiment). In this subtask, the directionality of the tweet sentiment is ignored. A negative tweet will be given the same score as a positive tweet with the same valence. 4. Sentiment intensity ordinal classification (V- oc): given tweet, predict sentiment ordinal intensity class from -3 (very negative) to 3 (very positive). 5. Emotion classification (E-c): given tweet, predict for each one of 11 emotions (anger, anticipation, disgust, fear, joy, love, opti- mism, pessimism, sadness, surprise, trust) whether the emotion is neutral (0) or present (1). The task is particularly challenging since E-c and EI-oc are completely new subtasks. Thus, no prior data or working models are available for compar- ison. The leaderboard is also not public during the competition. As shown in Table 1, taken from (Mohammad and Kiritchenko, 2018), the develop- ment sets are particularly small compared to the test sets, and the test sets are comparable in size to the training sets, so the model must generalize. For EI-oc and EI-reg, the development and test sets are also annotated separately from the train- ing sets. This impacts performance as our sys- tem would have placed 1st with average pearson score 0.755 on the WASSA 2017 task, in which the EI-reg train, development, and test data are an- notated in the same format. Furthermore, tweets are difficult to analyze due to the unstructuredness of its language (hashtags, emoticons, slang, mis- spellings, poor grammar). Previously …

Content truncated to 500 words