CX-ST-RNM at SemEval-2019 Task 3: Fusion of Recurrent Neural Networks Based on Contextualized and Static Word Representations for Contextual Emotion Detection

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 180–184 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 180 CX-ST-RNM at SemEval-2019 Task 3: Fusion of Recurrent Neural Networks Based on Contextualized and Static Word Representations for Contextual Emotion Detection Michał Perełkiewicz National Information Processing Institute al. Niepodległo´sci 188b, Warsaw, Poland michal.perelkiewicz@opi.org.pl Abstract In this paper, I describe a fusion model combi- ning contextualized and static word represen- tations for approaching the EmoContext task in the SemEval 2019 competition. The mo- del is based on two Recurrent Neural Ne- tworks, the first one is fed with a state-of-the- art ELMo deep contextualized word represen- tation and the second one is fed with a sta- tic Word2Vec embedding augmented with 10- dimensional affective word feature vector. The proposed model is compared with two base- line models based on a static word representa- tion and a contextualized word representation, separately. My approach achieved officially 0.7278 microaveraged F1 score on the test da- taset, ranking 47th out of 165 participants. 1 Introduction The EmoContext task in the Semantic Evaluation 2019 (SemEval 2019) competition focuses on the classification of textual dialogues i.e. a user short conversation with a bot, into ’happy’, ’sad’, ’an- gry’ and ’others’ sentiment classes. Understan- ding emotions in textual conversations is a chal- lenging task mainly because of an absence of others expression channels such as voice modula- tions and facial expressions usually accompanying a people conversation. In textual conversation de- termining the emotion of a given statement is very dependent on the context of previous statements. A sentiment detection field has been thoro- ughly analysed. The first attempts to manage this problem included mainly extracting hand-crafted features and knowledge-based systems (Balahur et al., 2011; Chaumartin, 2007; Joulin et al., 2016). Neural machine learning approaches, especially Recurrent Neural Networks (RNN) like LSTM and GRU and Convolutional Neural Networks (CNN), are effective tools for detecting sentiment from text and were widely used in this area (Tang et al., 2015; Liu et al., 2016; dos Santos and Gatti, 2014). Except that, one of the approaches employs Hierarchical Attention Networks (HAN) to deter- mining emotions from textual dialogues data (Sa- xena et al., 2018). In this work, I propose a deep neural fusion mo- del that combines two Bidirectional LSTM Recur- rent Neural Networks for detecting sentiments in textual dialogues. I use static Word2Vev word re- presentation and a contextualized ELMo word re- presentation to create a unified model. This archi- tecture of the model is inspired by the work pre- sented in (Gupta et al., 2017). The rest of the paper is structured as follows. Section 2 describes the proposed approach and word representations. The experiments and results are presented and discussed in Section 3. Finally, in the last section, the conclusions are presented. 2 Approach The following section provides details on prepro- cessing I used to normalize textual data provi- ded by the task organizers, the method to ma- nage unbalanced datasets problem and describes the …

Content truncated to 500 words