YUN-HPCC at SemEval-2019 Task 3: Multi-Step Ensemble Neural Network for Sentiment Analysis in Textual Conversation

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 360–364 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 360 YUN-HPCC at SemEval-2019 Task 3: Multi-Step Ensemble Neural Network for Sentiment Analysis in Textual Conversation Dawei Li, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Contact:Kunming, P.R. China xjzhang@ynu.edu.cn Abstract This paper describes our approach to the e- motion detection of Twitter textual conversa- tions based on deep learning. We analyze the syntax, abbreviations, and informal-writing of Twitter; and perform perfect data preprocess- ing on the data to convert them to normative text. We apply a multi-step ensemble strate- gy to solve the problem of extremely unbal- anced data in the training set. This is achieved by taking the GloVe and ELMo word vectors as input into a combination model with four different deep neural networks. The experi- mental results from the development dataset demonstrate that the proposed model exhibit- s a strong generalization ability. For evalua- tion on the test dataset, we integrated the re- sults using the stacking ensemble learning ap- proach and achieved competitive results. Ac- cording to the final official review, the results of our model achieved micro-F1 score of about 0.7588 on the final evaluation. 1 Introduction Over the past 10 years, short microblogging com- munication methods, such as Tweets and Weibo, have been widely adopted. Numerous emotion- s exist in the dialogue of texts, and there is great commercial value for the detection of such emo- tions. For example, in the customer service field, the feedback time limit is divided according to the emotion. Text conversation emotion detection is a chal- lenging issue without facial expressions and mood information being available. Moreover, the data of sadness, anger, and happiness in the current con- text, as well as the extremely unbalanced scale, natural language ambiguity, and rapidly growing online language, further exacerbate the challenges of sentiment detection. In this paper, we describe our work on Se- mEval 2019 Task 3, EmoContext: Contextu- al Emotion Detection in Text (Chatterjee et al., 2019). The main challenge of the task lies in imbalanced data distribution. Previously pro- posed methods for solving data imbalance include over-sampling, under-sampling (Weiss, 2004), and Synthetic Minority Oversampling Technique (S- MOTE) (Chawla et al., 2002). over-sampling is copying from a smaller number of categories, which may lead to over-fitting. Under-sampling discards potentially useful information, which can degrade the performance of the classifier (Drum- mond et al., 2003). SMOTE is down-sampling first, and then integration. We propose a multi- step integration approach similar to SMOTE for this task to alleviate the data imbalance problem. In the first step, we use five-fold cross-validation to train five sub-neural networks with different da- ta distributions. Thereafter, soft-voting is used for integrating the results from these sub-neural net- works. Soft-voting integration is applied because it not only alleviates the problem of unstable pre- diction results caused by data imbalance, but also improves the effective prediction accuracy of the single model. …

Content truncated to 500 words