THU-HCSI at SemEval-2019 Task 3: Hierarchical Ensemble Classification of Contextual Emotion in Conversation

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 345–349 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 345 THU-HCSI at SemEval-2019 Task 3: Hierarchical Ensemble Classification of Contextual Emotion in Conversation Xihao Liang, Ye Ma, Mingxing Xu Department of Computer Science & Technology Tsinghua University, Beijing, China (liangxh16, ma-y17)@mails.tsinghua.edu.cn, xumx@tsinghua.edu.cn Abstract In this paper, we describe our hierarchical ensemble system designed for the SemEval- 2019 task3, EmoContext. In our system, three sets of classifiers are trained for different sub- targets and the predicted labels of these base classifiers are combined through three steps of voting to make the final prediction. Effec- tive details for developing base classifiers are highlighted. Experiment results show that the ensemble approach manages to obtain better predictive performance in comparison with the base classifiers and our system has achieved the performance within the top 10 ranks in the final evaluation of EmoContext. 1 Introduction Sentiment analysis is a task to identify the emo- tion conveyed by written language (Balahur et al., 2011; Lee et al., 2018). With the popularization of the Internet and instant message applications, text has become one of the most familiar media by which people express their ideas and communicate with each other. Automatic emotion classification can help people and robots better understand the messages or comments written by the others and make proper responses, which makes this study field increasingly important (Sun et al., 2018). Deep learning approaches have achieved state- of-art performance in many recent studies of senti- ment analysis (Rodrigues do Carmo et al., 2017). Gupta et al. (2017) used an LSTM-based model to identify the emotion in 3-turn conversations on Twitter. For emotion detection on TV show transcripts, a sequence-based convolutional neu- ral network which can associate the information in the previous lines was designed by Zahiri and Choi (2017). Hazarika et al. (2018) proposed a conversational memory network based on both CNN (Lecun and Bengio, 1998) and gated recur- rent unit (Chung et al., 2014) to recognize emotion in dyadic dialogue videos, featuring its ability to manipulate the information of different speakers. These studies focused on the architecture of the neural networks, but the optimization of the classi- fication system based on the feature of the datasets are rarely discussed, which is crucial when dealing with specific real-world problems. In this paper, we describe our approach to SemEval-2019 task3, EmoContext (Chatterjee et al., 2019), which aims to encourage more re- search of contextual emotion detection in textual conversation. Datasets of 3-turn conversations are provided and the participating systems are re- quired to predict the contextual emotion of the last turn in the conversation: Happy, Sad, Angry or Others. The system performance is evaluated by the micro-averaged F1-score for Angry, Happy, and Sad (hereinafter referred to as AHS) on the given Test set. According to our observation of the dataset and single classifier’s performance on it, we design a hierarchical ensemble classification system, which is composed of three sets of base classifiers, and …

Content truncated to 500 words