SCIA at SemEval-2019 Task 3: Sentiment Analysis in Textual Conversations Using Deep Learning

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 297–301 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 297 SCIA at SemEval-2019 Task 3: Sentiment analysis in textual conversations using Deep Learning Zinedine Rebiai 1 zinedine.rebiai@epita.fr Antoine Debrenne 1 antoine.debrenne@epita.fr 1 EPITA Graduate School of Computer Science, France Simon Andersen 1 simon.andersen@epita.fr Victor Lafargue 1 victor.lafargue@epita.fr Abstract In this paper we present our submission for SemEval-2019 Task 3: EmoContext. The task consisted of classifying a textual dialogue into one of four emotion classes: happy, sad, an- gry or others. Our approach tried to improve on multiple aspects, preprocessing with an emphasis on spell-checking and ensembling with four different models: Bi-directional con- textual LSTM (BC-LSTM), categorical Bi- LSTM (CAT-LSTM), binary convolutional Bi-LSTM (BIN-LSTM) and Gated Recurrent Unit (GRU). On the leader-board, we submit- ted two systems that obtained a micro F1 score (F1µ) of 0.711 and 0.712. After the competi- tion, we merged our two systems with ensem- bling, which achieved a F1µ of 0.7324 on the test dataset. 1 Introduction Rapid progress in natural language processing with the rise of deep learning has brought increas- ing attention on tasks such as text classification and sentiment analysis. Most of the work in that field was made using social media due to the large amount of data available. The task addressed in this paper focuses on emotion detection within conversations from social media. The key point is that we need to take into account multiple speakers and capture a global emotion out of their conver- sation. It becomes a challenge when facing differ- ent users who each have a different way to express their emotions depending on their personalities. In a dialogue, users have an initial emotional state, and their mood will shift as the dialogue goes on. Therefore, the task of labelling a turn-based con- versation with the right emotion is even more chal- lenging. State of the art approaches consist of using lan- guage models (Vaswani et al., 2017) to pre-train the model on the general NLP task of language modeling before fine-tuning on specific tasks like classification or translation. The language model approach used by ULMFiT (Howard and Ruder, 2018), ELMO (Peters et al., 2018) and BERT (Devlin et al., 2018) was especially successful for this kind of tasks. For the specific task of emo- tion classification in textual conversation, Gupta et al. (2017) achieved a F1µ score of 0.7134 on the same dataset, using an architecture based on LSTM. For sentiment analysis, other successful approaches also used Bi-LSTM (Baziotis et al., 2017b) as well as transfer learning (Daval-Frerot et al., 2018). In this paper, we present two sub-systems that are composed of four deep-learning models (using Bi-LSTM, GRU and CNN). Those two sub-systems competed independently at SemEval- 2019 Task 3 (Chatterjee et al., 2019). After the fi- nal evaluation, we merged both sub-systems, tak- ing advantage of ensemble learning. The rest of the paper is organized as follows. Part 2 gives an overview …

Content truncated to 500 words