SemEval 2018 Task 2: Multilingual Emoji Prediction

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 24–33 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics SemEval 2018 Task 2: Multilingual Emoji Prediction Francesco Barbieri♦ Jose Camacho-Collados♣ Francesco Ronzano♥ Luis Espinosa-Anke♣ Miguel Ballesteros♠ Valerio Basile♥ Viviana Patti♥ Horacio Saggion♦ ♦ Large Scale Text Understanding Systems Lab, TALN. UPF. Barcelona, Spain ♣School of Computer Science and Informatics, Cardiff University ♠IBM Research, U.S ♥ Integrative Biomedical Informatics Group, GRIB, IMIM-UPF, Barcelona, Spain ♥Dipartimento di Informatica, University of Turin, Italy ♦♥{name.surname}@upf.edu, ♠miguel.ballesteros@ibm.com, ♣{espinosa-ankel,camachocolladosj}@cardiff.ac.uk, ♥{patti,basile}@di.unito.it Abstract This paper describes the results of the first shared task on Multilingual Emoji Prediction, organized as part of SemEval 2018. Given the text of a tweet, the task consists of predicting the most likely emoji to be used along such tweet. Two subtasks were proposed, one for English and one for Spanish, and participants were allowed to submit a system run to one or both subtasks. In total, 49 teams participated in the English subtask and 22 teams submitted a system run to the Spanish subtask. Evalua- tion was carried out emoji-wise, and the final ranking was based on macro F-Score. Data and further information about this task can be found at https://competitions. codalab.org/competitions/17344. 1 Introduction Emojis are small ideograms depicting objects, people, and scenes (Cappallo et al., 2015). Emojis are one of the main components of a novel way of communication emerging from the advent of so- cial media. They complement (usually) short text messages with a visual enhancement which is, as of now, a de-facto standard for online communi- cation (Barbieri et al., 2017). Figure 1 shows an example of a social media message displaying an emoji. Sometimes I think I wanna change the world... and I forget it just starts with changing me. Figure 1: Message from Twitter including a single red heart emoji. Emojis1 can be considered somehow an evolu- tion of character-based emoticons (Pavalanathan and Eisenstein, 2015), and currently they represent a widespread and pervasive global communication device largely adopted by almost any social media service and instant messaging platforms. Any system targeting the task of modeling so- cial media communication is expected to tackle the usage of emojis. In fact, their semantic load is sufficiently rich that oversimplifying them to sen- timent carriers or boosters would be to neglect the semantic richness of these ideograms, which in addition to mood ( lary references to food () include in their vocabu- ), sports ( ), scenery ( ), etc2. In general, however, effectively predict- ing the emoji associated with a piece of content may help to improve different NLP tasks (Novak et al., 2015), such as information retrieval, gener- ation of emoji-enriched social media content, sug- gestion of emojis when writing text messages or sharing pictures online. Given that emojis may also mislead humans (Barbieri et al., 2017; Miller et al., 2017), the automated prediction of emojis may help to achieve better language understand- ing. As a consequence, by modeling the semantics of emojis, we can …

Content truncated to 500 words