YNU-HPCC at SemEval-2018 Task 2: Multi-ensemble Bi-GRU Model with Attention Mechanism for Multilingual Emoji Prediction

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 459–465 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics YNU-HPCC at SemEval-2018 Task 2: Multi-ensemble Bi-GRU Model with Attention Mechanism for Multilingual Emoji Prediction Nan Wang, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Kunming, P.R. China Contact:xjzhang@ynu.edu.cn Abstract This paper describes our approach SemEval-2018 Task 2, which aims to predictto the most likely associated emoji, given a tweet in English or Spanish. We normalized text- based tweets during preprocessing, following which we utilized a bi-directional gated recurrent unit with an attention mechanism to build our base model. Multi-models with or without class weights were trained for the ensemble methods. We boosted models without class weights, and only strong boost classifiers were identified. In our system, not only was a boosting method used, but we also took advantage of the voting ensemble method to enhance our final system result. Our method demonstrated an obvious improvement of approximately 3% of the macro F1 score in English and 2% in Spanish. 1 Introduction As a novel means of enhancing the visual effect and meaning of short text messages, emojis are almost indispensable to each social platfor- m, such as Facebook, Twitter, and Instagram. These graphic facial expressions and other object symbols enrich the emotion the user wishes to express in a text-based message. Although they are significant as part of social messages, emojis have scarcely attracted attention from a natural language processing (NLP) standpoint. Notable exceptions include studies focused on emoji semantics and usages (Barbieri et al., 2017). However, the interplay between text- based messages and emojis remains virtually unexplored. The aim of SemEval-2018 task 2 (Barbieri et al., 2018) is to fill this gap by providing all participants with a large set of text- based tweets and their related emojis, which were extracted from original tweet messages, in order to determine the connection between text words and emojis. In recent years, an increasing amount of re- search work has been conducted on the sentiment analysis of tweets. Emojis have always played an important role in the sentiment polarity of tweets. Novak et al. (2015) proposed a sentiment map of the 751 most frequently used emojis, by computing the sentiment emoji from the sentiment of tweets in which they occurred, and they determined a significant difference in the sentiment distribution of tweets with and without emojis. As opposed to sentiment polarity prediction of tweets, we further investigated potential tweet emojis in this task, evoked by the text part. Neural networks involving attention mecha- nisms have been studied extensively in the image processing and NLP fields, and have demonstrated remarkable results, particularly convolutional neu- ral networks (CNNs) (Collobert et al., 2011) and long short-term memories (LSTMs) (Hochreiter and Schmidhuber, 1997). Cliche (2017) assem- bled several CNNs and LSTMs and achieved first ranking in all of five English subtasks. Raffel and Ellis (2015) proposed a feed-forward network model with attention, which selects the most important …

Content truncated to 500 words