FOI DSS at SemEval-2018 Task 1: Combining LSTM States, Embeddings, and Lexical Features for Affect Analysis

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 109–115 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics FOI DSS at SemEval-2018 Task 1: Combining LSTM States, Embeddings, and Lexical Features for Affect Analysis Maja Karasalo Mattias Nilsson Magnus Rosell Ulrika Wickenberg Bolin FOI - Swedish Defence Research Agency {majkar, matnil, magros, ulrwic}@foi.se Abstract This paper describes the system used and re- sults obtained for team FOI DSS at SemEval- 2018 Task 1: Affect In Tweets. The team participated in all English language subtasks, with a method utilizing transfer learning from LSTM nets trained on large sentiment datasets combined with embeddings and lexical fea- tures. For four out of five subtasks, the sys- tem performed in the range of 92-95% of the winning systems, in terms of the competi- tion metrics. Analysis of the results suggests that improved pre-processing and addition of more lexical features may further elevate per- formance. 1 Introduction In the field of automatic emotion detection, many contributions consider the issue of detecting pres- ence of emotions (Liu, 2012). The task of de- tecting intensity of emotion in a given text is less studied, but is relevant to many applications in fields such as e.g., brand management, public health, politics, and disaster handling (Moham- mad, 2016). When developing prediction sys- tems, access to suitably annotated data is criti- cal. Most annotated emotion and affect datasets are categorical, but examples of sets annotated with intensity or degree of emotional content in- clude EmoBank (Buechel and Hahn, 2017a,b), AFINN (Nielsen, 2011), the Pietro Facebook post set (Preot¸iuc-Pietro et al., 2016), and the Warriner- Kuperman set (Warriner et al., 2013). For tweets, the Tweet Emotion Intensity Dataset (Mohammad and Bravo-Marquez, 2017) has recently been pub- lished, with more than 7000 tweets annotated with emotion category and intensity. This paper describes methods used and results achieved with the FOI DSS contribution to the five subtasks for English tweets of SemEval 2018 Task 1: Affect in Tweets (Mohammad et al., 2018). The paper is organized as follows. A descrip- tion of Task 1 is provided in Section 2. Sec- tion 3 discusses the provided datasets. Section 4 describes the methods and system used to produce predictions of scores and labels for all subtasks. In Sections 5 and 6 results are presented and ana- lyzed, and suggestions for improvements are out- lined. Finally, concluding remarks are found in Section 7. 2 Task formulation Task 1 consisted of five subtasks, all regarding es- timation of the mental state of a tweeter, based on the tweeted text. Valence1 intensity, as well as emotion, and emotion intensity classification, were covered. The subtasks are summarized be- low: 1. Emotion intensity regression (EI-reg): For a given tweet and emotion2, determine the in- tensity of the emotion as a score ∈ [0, 1]. 2. Emotion intensity, ordinal classification (EI-oc): For a given tweet and emotion2, classify the tweet into one of four ordinal classes of intensity. 3. Valence regression (V-reg): For a given tweet, …

Content truncated to 500 words