CLaC Lab at SemEval-2019 Task 3: Contextual Emotion Detection Using a Combination of Neural Networks and SVM

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 153–158 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 153 CLaC Lab at SemEval-2019 Task 3: Contextual Emotion Detection Using a Combination of Neural Networks and SVM Elham Mohammadi, Hessam Amini and Leila Kosseim Computational Linguistics at Concordia (CLaC) Lab Department of Computer Science and Software Engineering Concordia University, Montr´eal, Qu´ebec, Canada first.last@concordia.ca Abstract This paper describes the CLaC Lab system at SemEval 2019, Task 3 (EmoContext), which focused on the contextual detection of emo- tions in a dataset of 3-round dialogues. For our final system, we used a neural network with pretrained ELMo word embeddings and POS tags as input, GRUs as hidden units, an atten- tion mechanism to capture representations of the dialogues, and an SVM classifier which used the learned network representations to perform the task of multi-class classification. This system yielded a micro-averaged F1 score of 0.7072 for the three emotion classes, im- proving the baseline by approximately 12%. 1 Introduction Automatic emotion detection has been the focus of much research in a variety of fields, includ- ing emotion detection based on images (Rao et al., 2019), speech signals (Davletcharova et al., 2015), electroencephalography (EEG) signals (Acker- mann et al., 2016), and texts (Tafreshi and Diab, 2018). With the advent of social media, emotion de- tection from text has been used to track bloggers’ mental health and has been explored using differ- ent techniques, such as lexicon-based approaches and machine learning (Canales and Mart´ınez- Barco, 2014). Lexicon-based approaches include keyword-based and ontological approaches. In a keyword-based approach (e.g. Ma et al., 2005), a specific set of opinion terms and their Word- Net synonyms and antonyms are used to determine the emotion class of the text. On the other hand, ontology-based approaches (e.g. Sykora et al., 2013) try to detect emotions by taking into ac- count the knowledge of concepts, the intercon- nection between them, and their final emotional impact. To achieve better generalization, the re- sources used in lexicon-based approaches can be employed as input features to supervised or unsu- pervised machine learning models. The drawback with supervised machine learn- ing approaches lies in the need for a large corpus of labelled data. Abdul-Mageed and Ungar (2017) collected a labelled twitter dataset of 1/4 billion tweets, spanning over 8 primary emotions, using distant supervision, i.e. collecting tweets with emotion hashtags that can be used as labels. Then using a gated recurrent neural network for classi- fication, they achieved an accuracy of 95.68%, su- perior to the best previously published results by Volkova and Bachrach (2016). Although much research has focused on the de- tection of emotions in tweets and blog posts (e.g. Mohammad, 2012; Desmet and Hoste, 2013; Liew and Turtle, 2016), emotion detection in dialogues, as well as in single utterances has received very little attention. This topic can have significant im- pact for the development of social chatbots, with the aim of creating an emotional connection be- tween a user …

Content truncated to 500 words