Multi-resolution Annotations for Emoji Prediction

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6684–6694, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6684 Multi-resolution Annotations for Emoji Prediction Weicheng Ma1, Ruibo Liu2, Lili Wang3, and Soroush Vosoughi4 Minds, Machines, and Society Group Department of Computer Science, Dartmouth College 1,2,3{first.last.gr}@dartmouth.edu 4soroush.vosoughi@dartmouth.edu Abstract Emojis are able to express various linguistic components, including emotions, sentiments, events, etc. Predicting the proper emojis as- sociated with text provides a way to summa- rize the text accurately, and it has been proven to be a good auxiliary task to many Natural Language Understanding (NLU) tasks. La- bels in existing emoji prediction datasets are all passage-based and are usually under the multi-class classification setting. However, in many cases, one single emoji cannot fully cover the theme of a piece of text. It is thus useful to infer the part of text related to each emoji. The lack of multi-label and aspect- level emoji prediction datasets is one of the bottlenecks for this task. This paper annotates an emoji prediction dataset with passage-level multi-class/multi-label, and aspect-level multi- class annotations. We also present a novel an- notation method with which we generate the aspect-level annotations. The annotations are generated heuristically, taking advantage of the self-attention mechanism in Transformer networks. We validate the annotations both au- tomatically and manually to ensure their qual- ity. We also benchmark the dataset with a pre- trained BERT model. 1 Introduction Emojis have become crucial components of writ- ten language. Emojis were initially designed to express emotions or feelings, e.g., for a smiley face, and they have grown to be a large family of over 2,000 icons over the years which can express not only emotions but a wide range of objects or actions, e.g., for a gift and for celebrations. Compared to words, emojis have the merit of pre- serving information more densely. For example, carries the same meaning as the phrase “laughing with tears in eyes”. Additionally, the byte-level encoding of subtle linguistic expressions makes it easier to discriminate complicated feelings, e.g., the bond between and is clearly weaker than their phrasal explanations “laughing with tears in eyes” and “crying loudly” due to the similarity be- tween “tear” and “crying”. These characteristics of emojis aid in accurate summarization of text, thus benefiting natural language understanding (NLU) tasks. Felbo et al. (2017) define the emoji prediction task by finding the most appropriate emoji(s) sum- marizing a piece of text. They also show with experiments that language representations learned on the emoji prediction task can boost the perfor- mance of emotion recognition, sentiment analysis, and sarcasm detection tasks. Consequently, using emoji prediction as a bridge to solve other natu- ral language processing (NLP) tasks appears to be effective and promising. However, the emoji prediction task is yet far from being well estab- lished. First and foremost, as a classification task, there is not a set of labels agreed upon by previ- ous research. To the best of our knowledge, all the existing papers on …

Content truncated to 500 words