EmojiIt at SemEval-2018 Task 2: An Effective Attention-Based Recurrent Neural Network Model for Emoji Prediction with Characters Gated Words

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 423–427 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics EmojiIt at SemEval-2018 Task 2: An Effective Attention-Based Recurrent Neural Network Model for Emoji Prediction with Characters Gated Words Shiyun Chen, Maoquan Wang, Liang He Department of Computer Science and Technology East China Normal University Shanghai, P.R.China 200241 51174506002@stu.ecnu.edu.cn, maoquanwang@ica.stc.sh.cn, lhe@cs.ecnu.edu.cn Abstract This paper presents our single model to Sub- task 1 of SemEval 2018 Task 2: Emoji Predic- tion in English. In order to predict the emo- ji that may be contained in a tweet, the basic model we use is an attention-based recurrent neural network which has achieved satisfacto- ry performs in Natural Language processing. Considering the text comes from social media, it contains many discrepant abbreviations and online terms, we also combine word-level and character-level word vector embedding to bet- ter handling the words not appear in the vo- cabulary. Our single model1 achieved 29.50% Macro F-score in test data and ranks 9th a- mong 48 teams. 1 Introduction SemEval-2018 shared task 2 (Barbieri et al., 2018) provides a platform for us to explore the relation- ship between text and emoji in Twitter. The par- ticipants are expected to predict, given a tweet in English or Spanish, its most likely associated e- moji. As the number of frequently-used emojis up to 20, this is also a multi-category classification task. Overall, both the prediction of emoji and the multi-category classification have undoubtedly in- creased the difficulty of task 2. Before choosing a suitable model, we first an- alyze the data characteristics in detail. For each emoji, we count the total number of all tweets words under this emoji and enumerate the top 10 meaningful words with the highest frequency. And we can explore the following observations from the statistics: • In almost all emojis, the first three words that appear frequently include “@user” suggest- ing that Twitter is a highly interactive social 1https://github.com/wwmmqq/SemEval-2018-Task-2- Multilingual-Emoji-Prediction networking site (all the person names uni- fied as “users” during the data preprocess- ing). Such a tweet should include an unwrit- ten but important context information. • Because people do not always have the same understanding of emojis, some words can e- voke more than one emoji. The greater the repetition of high-frequency words among d- ifferent emojis, the harder it is to distinguish among these tweets. • There are also some easily recognizable words. They are just common words un- der a certain emoji, almost do not appear under others, such as “lit” and “fire” under ; “photo” under ; “usa”, “america” and “vote” under ; “beach” and “summer” un- der . • When glancing over the data, we find that tweets have the following characteristics: a) Non-standard language, some discrepant shorthand or Internet jargon causes the model to get confused about the word; b) Misspelled words, although the human can easily real- ize the correct words, but if the model takes word-level as the processing unit, it cannot …

Content truncated to 500 words