Deep Learning Brasil - NLP at SemEval-2020 Task 9: Sentiment Analysis of Code-Mixed Tweets Using Ensemble of Language Models

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1233–1238 Barcelona, Spain (Online), December 12, 2020. 1233 Deep Learning Brasil - NLP at SemEval-2020 Task 9: Sentiment Analysis of Code-Mixed Tweets Using Ensemble of Language Models Manoel Ver´ıssimo dos Santos Neto INF - Federal University of Goi´as verissimo.manoel@gmail.com Ayrton Denner da Silva Amaral INF - Federal University of Goi´as ayrtondenner2013@gmail.com N´adia F. F. da Silva INF - Federal University of Goi´as nadia@inf.ufg.br Anderson da Silva Soares INF - Federal University of Goi´as anderson@inf.ufg.br Abstract In this paper, we describe a methodology to predict sentiment in code-mixed tweets (hindi-english). Our team called verissimo.manoel in CodaLab1 developed an approach based on an ensemble of four models (MultiFiT, BERT, ALBERT, and XLNET). The final classification algorithm was an ensemble of some predictions of all softmax values from these four models. This architecture was used and evaluated in the context of the SemEval 2020 challenge (task 9), and our system got 72.7% on the F1 score. 1 Introduction It is a common tendency among multilingual people who are non-native English speakers to code-mix in their speech using English-based phonetic typing. This linguistic phenomenon, particularly in social media like Twitter2, poses a great challenge to the conventional Natural Language Processing (NLP) study area. Within the context of the Sentiment Analysis, the study of the phenomenon of code-mixed language is important to the research community because this behavior is more common today. The interest in this area has grown due to the volume of data that social networks generate, and also by the value that this information has to understand people opinions when they are expressed in written texts. In this paper, we explain our methodology to predict sentiment in tweets, describing how our method is based on a combination of the latest language models, and also how such models contributed to a great advance in this task. This configuration was employed and evaluated in the SemEval 2020 challenge (task 9), in which the goal is to predict the sentiment in code-mixed texts written in English and Hindi languages of a tweet (Patwa et al., 2020). The models used in this combination are: MultiFiT (Eisenschlos et al., 2019) that an evolution of ULMFiT (Howard and Ruder, 2018), BERT (Devlin et al., 2018), ALBERT (Lan et al., 2019) and XLNet (Yang et al., 2019b). This work is organized as follows: Section 2 explains some related works, Section 4 describes the dataset used, Section 3 addresses the methodology applied in the task, Section 5 presents the results, and finally Section 6 expose our final considerations as well as possible future works. 2 Related Works Sentiment Analysis in Twitter has been considered as a very important task from various academic and commercial perspectives. Many companies use the data on Twitter to decide about marketing and business decisions. A challenge is to apply Sentiment Analysis in texts written in two different languages like English and Hindi. This happens because some people around the world speak two or more languages …

Content truncated to 500 words