THU_NGN at SemEval-2018 Task 2: Residual CNN-LSTM Network with Attention for English Emoji Prediction

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 410–414 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics THU NGN at SemEval-2018 Task 2: Residual CNN-LSTM Network with Attention for English Emoji Prediction Chuhan Wu1, Fangzhao Wu2, Sixing Wu1, Zhigang Yuan1, Junxin Liu1 and Yongfeng Huang1 1Tsinghua National Laboratory for Information Science and Technology, Department of Electronic Engineering, Tsinghua University Beijing 100084, China 2Microsoft Research Asia {wuch15,wu-sx15,yuanzg14,ljx16,yfhuang}@mails.tsinghua.edu.cn wufangzhao@gmail.com Abstract Emojis are widely used by social media and social network users when posting their mes- sages. It is important to study the relation- ships between messages and emojis. Thus, in SemEval-2018 Task 2 an interesting and chal- lenging task is proposed, i.e., predicting which emojis are evoked by text-based tweets. We propose a residual CNN-LSTM with attention (RCLA) model for this task. Our model com- bines CNN and LSTM layers to capture both local and long-range contextual information for tweet representation. In addition, attention mechanism is used to select important com- ponents. Besides, residual connection is ap- plied to CNN layers to facilitate the training of neural networks. We also incorporated addi- tional features such as POS tags and sentiment features extracted from lexicons. Our model achieved 30.25% macro-averaged F-score in the first subtask (i.e., emoji prediction in En- glish), ranking 7th out of 48 participants. 1 Introduction Emojis such as and are widely used in so- cial media and social network messages such as tweets. They are frequently combined with plain texts to visually complement the meaning of a message and convey various opinions and emo- tions (Novak et al., 2015; Barbieri et al., 2017). Social media platforms such as Twitter has ac- cumulated a large number of emoji-incorporated messages. Analyzing the relationships between the textual message and emojis has many potential applications, such as emoji recommendation, au- tomatic emoji-enriched message generation, and accurate sentiment analysis of social media mes- sages (Barbieri et al., 2017). However, the research on the relationships be- tween textual message and emojis is limited. Ex- isting studies on emojis mainly focus on analyzing the semantics, usage or sentiment of emojis (Aoki and Uchida, 2011; Barbieri et al., 2016a,b,c; Ljubeˇsi´c and Fiˇser, 2016; Novak et al., 2015). For example, Barbieri et al. (2016b) explored the meaning and usage of emojis across different lan- guages. Wijeratneet al. (2017) proposed to uti- lize the emoji sense definitions to improve the per- formance of emoji embedding model. However, these approaches cannot reveal the interplay be- tween plain texts and emojis. In order to fill this gap, Barbieri et al. (2017) proposed a novel task to predict which emojis are evoked by text-based tweets. For example, given a tweet message “Love my coworkers ! @user”, a system is required to predict that emoji is associated with this tweet. As an extension, the SemEval-2018 Task 21 aims to predict emojis for English and Spanish tweets (Barbieri et al., 2018). Given a plain tweet message without emoji, systems are required to predict which emoji is evoked by …

Content truncated to 500 words