IIT Gandhinagar at SemEval-2019 Task 3: Contextual Emotion Detection Using Deep Learning

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 236–240 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 236 IIT Gandhinagar at SemEval-2019 Task 3: Contextual Emotion Detection Using Deep Learning Arik Pamnani∗, Rajat Goel∗, Jayesh Choudhari, Mayank Singh IIT Gandhinagar Gujarat, India {arik.pamnani,rajat.goel,choudhari.jayesh,singh.mayank}@iitgn.ac.in Abstract Recent advancements in Internet and Mobile infrastructure have resulted in the development of faster and efficient platforms of communi- cation. These platforms include speech, facial and text-based conversational mediums. Ma- jority of these are text-based messaging plat- forms. Development of Chatbots that automat- ically understand latent emotions in the tex- tual message is a challenging task. In this pa- per, we present an automatic emotion detec- tion system that aims to detect the emotion of a person textually conversing with a chatbot. We explore deep learning techniques such as CNN and LSTM based neural networks and outper- formed the baseline score by 14%. The trained model and code are kept in public domain. 1 Introduction In recent times, text has become a preferred mode of communication (Reporter, 2012; ORTUTAY, 2018) over phone/video calling or face-to-face communication. New challenges and opportuni- ties accompany this change. Identifying sentiment from text has become a sought after research topic. Applications include detecting depres- sion (Wang et al., 2013) or teaching empathy to chatbots (WILSON, 2016). These applications leverage NLP for extracting sentiments from text. On this line, SemEval Task 3: EmoContext (Chatterjee et al., 2019) challenges participants to identify Contextual Emotions in Text. Challenges: The challenges with extracting sentiments from text are not only limited to the use of slang, sarcasm or multiple languages in a sen- tence. There is also a challenge which is presented by the use of non-standard acronyms specific to in- dividuals and others which are present in the task’s dataset 2. ∗Equal Contribution Existing work: For sentiment analysis, most of the previous year’s submissions focused on neu- ral networks (Nakov et al., 2016). Teams exper- imented with Recurrent Neural Network (RNN) (Yadav, 2016) as well as Convolutional Neu- ral Network (CNN) based models (Ruder et al., 2016). However, some top ranking teams also used (Giorgis et al., 2016) classic machine learn- ing models. Aiming for the best system, we started with classical machine learning algorithms like Support Vector Machine (SVM) and Logistic Re- gression (LR). Based on the findings from them we moved to complex models using Long Short- Term Memory (LSTM), and finally, we experi- mented with CNN in search for the right system. Our contribution: In this paper, we present models to extract emotions from text. All our models are trained using only the dataset provided by EmoContext organizers. The evaluation metric set by the organizers is micro F1 score (referred as score in rest of the paper) on three {Happy, Sad, Angry} out of the four labels. We experi- mented by using simpler models like SVM, and Logistic regression but the score on dev set was below 0.45. We then worked with a CNN model and …

Content truncated to 500 words