CrystalFeel at SemEval-2018 Task 1: Understanding and Detecting Emotion Intensity using Affective Lexicons

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 256–263 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics CrystalFeel at SemEval-2018 Task 1: Understanding and Detecting Emotion Intensity using Affective Lexicons Raj Kumar Gupta and Yinping Yang∗ Institute of High Performance Computing (IHPC) Agency for Science, Technology and Research (A*STAR), Singapore {gupta-rk, yangyp}@ihpc.a-star.edu.sg Abstract While sentiment and emotion analysis has re- ceived a considerable amount of research at- tention, the notion of understanding and de- tecting the intensity of emotions is relatively less explored. This paper describes a system developed for predicting emotion intensity in tweets. Given a Twitter message, CrystalFeel uses features derived from parts-of-speech, n- grams, word embedding, and multiple affec- tive lexicons including Opinion Lexicon, Sen- tiStrength, AFFIN, NRC Emotion & Hash Emotion, and our in-house developed EI Lexi- cons to predict the degree of the intensity asso- ciated with fear, anger, sadness, and joy in the tweet. We found that including the affective lexicons-based features allowed the system to obtain strong prediction performance, while revealing interesting emotion word-level and message-level associations. On gold test data, CrystalFeel obtained Pearson correlations of .717 on average emotion intensity and of .816 on sentiment intensity. 1 Introduction While humans experience emotions every day, the degree of one’s emotions varies from one experi- ence to another. To date, a vast majority of NLP and computational linguistics research deals with ground truth data constructed through the assign- ment of discrete labels to text messages by anno- tators. Conventionally, sentiment analysis seeks to determine the valence (positive, negative or neu- tral) of the feelings and opinions that annotators can recognize in a text message (Hu and Liu, 2004; Pang and Lee, 2008; Socher et al., 2013). Emo- tion classification, a closely related task, typically seeks to predict the presence or absence of an emo- tion, i.e., if there is joy or no joy, anger or no ∗Both authors contributed to this research equally. Cor- respondence should be sent to yangyp@ihpc.a-star.edu.sg. anger, fear or no fear, in a particular message (Alm et al., 2005; Aman and Szpakowicz, 2007; Wen and Wan, 2014). The detection of emotion inten- sity along a continuous scale is a relatively less explored task. One of the key reasons for the lack of work on detecting emotion intensity is plausibly at- tributable to the difficulty in measuring the very concept of emotion intensity. As highlighted in prior research, the question “how intense was your emotional experience on a scale of 1 to 10?” can- not generate reliable responses even for the same emotion type (Frijda et al., 1992). For example, asking people to respond to “how intense was your fear towards getting rejected” and “how intense was your fear towards receiving a medical test re- sult” would lead to inconsistent answers across the same annotators at different times, as well as across different annotators. Because of the lack of a clear reference point, it is nearly impossible to construct ground truth datasets with adequate reli- …

Content truncated to 500 words