Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1615–1625 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm Bjarke Felbo1, Alan Mislove2, Anders Søgaard3, Iyad Rahwan1, Sune Lehmann4 1Media Lab, Massachusetts Institute of Technology 2College of Computer and Information Science, Northeastern University 3Department of Computer Science, University of Copenhagen 4DTU Compute, Technical University of Denmark Abstract NLP tasks are often limited by scarcity of manually annotated data. In social me- dia sentiment analysis and related tasks, researchers have therefore used binarized emoticons and specific hashtags as forms of distant supervision. Our paper shows that by extending the distant supervision to a more diverse set of noisy labels, the models can learn richer representations. Through emoji prediction on a dataset of 1246 million tweets containing one of 64 common emojis we obtain state-of-the- art performance on 8 benchmark datasets within emotion, sentiment and sarcasm de- tection using a single pretrained model. Our analyses confirm that the diversity of our emotional labels yield a performance improvement over previous distant super- vision approaches. 1 Introduction A variety of NLP tasks are limited by scarcity of manually annotated data. Therefore, co-occurring emotional expressions have been used for dis- tant supervision in social media sentiment anal- ysis and related tasks to make the models learn useful text representations before modeling these tasks directly. For instance, the state-of-the-art ap- proaches within sentiment analysis of social me- dia data use positive/negative emoticons for train- ing their models (Deriu et al., 2016; Tang et al., 2014). Similarly, hashtags such as #anger, #joy, #happytweet, #ugh, #yuck and #fml have in pre- vious research been mapped into emotional cate- gories for emotion analysis (Mohammad, 2012). Distant supervision on noisy labels often en- ables a model to obtain better performance on the target task. In this paper, we show that extend- ing the distant supervision to a more diverse set of noisy labels enables the models to learn richer rep- resentations of emotional content in text, thereby obtaining better performance on benchmarks for detecting sentiment, emotions and sarcasm. We show that the learned representation of a single pretrained model generalizes across 5 domains. Table 1: Example sentences scored by our model. For each text the top five most likely emojis are shown with the model’s probability estimates. I love mom's cooking 49.1% 8.8% 3.1% 3.0% 2.9% I love how you never reply back.. 14.0% 8.3% 6.3% 5.4% 5.1% I love cruising with my homies 34.0% 6.6% 5.7% 4.1% 3.8% I love messing with yo mind!! 17.2% 11.8% 8.0% 6.4% 5.3% I love you and now you're just gone.. 39.1% 11.0% 7.3% 5.3% 4.5% This is shit 7.0% 6.4% 6.0% 6.0% 5.8% This is the shit 10.9% 9.7% 6.5% 5.7% 4.8% Emojis are not always a direct labeling of emo- tional content. For instance, a positive emoji may serve to disambiguate an ambiguous sentence or to complement an otherwise relatively negative …

Content truncated to 500 words