DL Team at SemEval-2018 Task 1: Tweet Affect Detection using Sentiment Lexicons and Embeddings

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 172–176 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics DL Team at SemEval-2018 Task 1: Tweet Affect Detection using Sentiment Lexicons and Embeddings Dmitry Kravchenko Ben-Gurion University of the Negev / Israel to.dmitry.kravchenko@gmail.com Lidia Pivovarova University of Helsinki / Finland lidia.pivovarova@cs.helsinki.fi Abstract The paper describes our approach for SemEval-2018 Task 1: Affect Detection in Tweets. We perform experiments with manually compelled sentiment lexicons and word embeddings. We test their performance on twitter affect detection task to determine which features produce the most informative representation of a sentence. We demon- strate that general-purpose word embeddings produces more informative sentence repre- sentation than lexicon features. However, combining lexicon features with embeddings yields higher performance than embeddings alone. 1 Introduction The paper describes our approach for SemEval- 2018 Task 1: Affect Detection in Tweets (Moham- mad et al., 2018). The research question we address in this paper is what are the best features for tweet affect de- tection. Our solution uses two types of features: lexicon features obtained from manually compiled emotion lexicons, and word embeddings built un- supervisedly from large corpora. We use well established lexicons, namely DepecheMood and Vader Sentiment, and most popular Word embed- dings, namely GloVe and Google News. We sys- tematically compare all features on two subtasks and demonstrate that even though lexicon features produce unsatisfactory results in isolation, they significantly improve an algorithm performance when combined with more general embeddings. In addition, we demonstrate that special treat- ment of Twitter hash-tags also improves the algo- rithm performance. 2 Tasks and Data The paper addresses three subtasks: Task Train Dev Test EI-reg all emotions 7102 1464 71816 anger 1701 388 17939 fear 2252 389 17923 joy 1616 290 18042 sadness 1533 397 17912 V-reg 1181 449 17874 E-c 6838 886 3259 Table 1: Training, development and test set split for three subtasks • EI-reg—an emotion intensity regression task: Given a tweet and an emotion E, de- termine the intensity of E that best repre- sents the mental state of the tweeter—a real- valued score between 0 (no E at all) and 1 (the highest magnitude of E); separate datasets are provided for fear, sadness, anger, and joy. • V-reg—a sentiment intensity regression task: Given a tweet, determine the intensity of sen- timent or valence (V) that best represents the mental state of the tweeter—a real-valued score between 0 (most negative) and 1 (most positive). • E-c—an emotion classification task: Given a tweet, classify it as ’neutral or no emotion’ or as one, or more, of eleven given emo- tions that best represent the mental state of the tweeter: trust, sadness, disgust, fear, opti- mism, love, joy, pessimism, anticipation, sur- prise, and anger. We use English data for all three subtasks. The train, development and test set sizes are shown in Table 1. More details on the data can be found in the task organizers’ paper (Mohammad and Kir- itchenko, 2018). 172 3 Approach 3.1 …

Content truncated to 500 words