RIDDL at SemEval-2018 Task 1: Rage Intensity Detection with Deep Learning

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 358–363 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics RIDDL at SemEval-2018 Task 1: Rage Intensity Detection with Deep Learning Venkatesh Elango, Karan Uppal Bloomberg New York, NY, USA {velango,kuppal8}@bloomberg.net Abstract We present our methods and results for af- fect analysis in Twitter developed as a part of SemEval-2018 Task 1, where the sub-tasks in- volve predicting the intensity of emotion, the intensity of sentiment, and valence for tweets. For modeling, though we use a traditional LSTM network, we combine our model with several state-of-the-art techniques to improve its performance in a low-resource setting. For example, we use an encoder-decoder network to initialize the LSTM weights. Without any task specific optimization we achieve compet- itive results (macro-average Pearson correla- tion coefficient 0.696) in the El-reg task. In this paper, we describe our development strat- egy in detail along with an exposition of our results. 1 Introduction Sentiment analysis is a technique to classify doc- uments based on the polarity of opinion expressed by the author of the document (Pang et al., 2002). Traditionally this involved extracting coarse sen- timent (positive, negative, or neutral) from doc- uments such as news articles, product or movie reviews (Wiebe et al., 2005; Hu and Liu, 2004). In order to get a fine grained view of the opinion, sentiment analysis was applied at the sentence and phrase level (Yu and Hatzivassiloglou, 2003; Wil- son et al., 2005). With the advent of social me- dia, Twitter in particular, sentiment towards a wide range of topics could be extracted at a much larger scale than before. This however came with its own set of problems, viz., a lack of proper grammatical structure, prevalence of slang, acronyms, and mis- spellings (Jansen et al., 2009; Barbosa and Feng, 2010). SemEval tasks have provided a curated test- ing environment for analysis on Twitter data with tasks to quantify sentiment on two-point, three- point, and five-point scales (Nakov et al., 2016; Rosenthal et al., 2017). While a finer gradation in polarity of the text could be inferred by intro- ducing more nuanced categories, it faces the prob- lem of needing to collect more labeled data as the number of classes increase. Therefore, this approach requires that the intensity of the senti- ment expressed be measured on a continuous scale rather than through discrete categories. SemEval 2018 Task 1 takes a novel step in this direction by introducing tasks for predicting intensity of emo- tion, or sentiment expressed (Mohammad et al., 2018). We participated in SemEval-2018 Task-1 (El- Reg, V-reg, V-oc) and our contributions through this paper are as follows: • We present an LSTM network combined with known state-of-the-art techniques to improve performance on low-resource setting tasks. • The proposed model requires no task specific hyper-parameter tuning. • We perform error analysis of our model to obtain a better understanding of strengths and weaknesses of a deep learning-based ap- proach for these tasks and propose improve- ments. …

Content truncated to 500 words