UMDuluth-CS8761 at SemEval-2018 Task 2: Emojis: Too many Choices?

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 400–404 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics UMDuluth-CS8761 at SemEval-2018 Task 2: Emojis: Too many Choices? Dennis Asamoah Owusu & Jonathan Beaulieu Department of Computer Science University of Minnesota Duluth Duluth, MN 55812 USA {asamo012,beau0307}@d.umn.edu https://github.com/derpferd/semeval-2018-task2/ Abstract In this paper, we present our system for assign- ing an emoji to a tweet based on the text. Each tweet was originally posted with an emoji which the task providers removed. Our task was to decide out of 20 emojis, which origi- nally came with the tweet. Two datasets were provided - one in English and the other in Spanish. We treated the task as a standard clas- sification task with the emojis as our classes and the tweets as our documents. Our best per- forming system used a Bag of Words model with a Linear Support Vector Machine as its’ classifier. We achieved a macro F1 score of 32.73% for the English data and 17.98% for the Spanish data. 1 Introduction An AI system that can associate text with appro- priate emojis could be useful for generating con- tent that is sparkled with emojis among other uses (Barbieri et al., 2017). Given only the text from a tweet in English or Spanish, the SemEval (Bar- bieri et al., 2018) task was to determine the emoji that was in the original tweet. To learn how users associate emojis with text, a dataset comprised of 489,609 tweets in English and 98,289 tweets in Spanish was provided. Each tweet had a corre- sponding label representing the emoji that was in the tweet. The labels were assigned based on the frequency of the emoji, 0 being assigned to the most frequent emoji. The total number of labels was 20 for the English data and 19 for the Spanish data. We classified the tweets, our documents, by their emojis, our classes. We viewed the emojis as approximations of the sentiment expressed in the text. For our baseline, we implemented a Bag of Words model using a Bernoulli Naive Bayes clas- sifier. We analyzed the results and used the in- sights to implement our final system, which used a Linear Support Vector machine for classification and also used a Bag of Words model to represent each document. This system performed better than our baseline by ~3.5 percentage points for our En- glish data and ~1.5 percentage points for our Span- ish data. It also performed better than several neu- ral network models we experimented with. Our macro F1 score were 32.73% and 17.98% for our English data and Spanish data respectively. 2 Baseline For our Baseline, we used a Bag of Words model (BOW) with a Bernoulli Naive Bayes Classi- fier. We also implemented a Most Frequent Class Model (MFC) and a Random Model (RAND) to help draw insights from our baseline. The results of these models for the English and Spanish data are shown in Table 1 and Table 2 …

Content truncated to 500 words