CS-Embed at SemEval-2020 Task 9: The Effectiveness of Code-switched Word Embeddings for Sentiment Analysis

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 922–927 Barcelona, Spain (Online), December 12, 2020. 922 CS-Embed at SemEval-2020 Task 9: The effectiveness of code-switched word embeddings for sentiment analysis Frances A. Laureano De Leon University of Birmingham United Kingdom laureanofa@gmail.com Florimond Gu´eniat Birmingham City University United Kingdom florimond.gueniat@bcu.ac.uk Harish Tayyar Madabushi University of Birmingham United Kingdom H.TayyarMadabushi.1@bham.ac.uk Abstract The growing popularity and applications of sentiment analysis of social media posts has naturally led to sentiment analysis of posts written in multiple languages, a practice known as code- switching. While recent research into code-switched posts has focused on the use of multilingual word embeddings, these embeddings were not trained on code-switched data. In this work, we present word-embeddings trained on code-switched tweets, specifically those that make use of Spanish and English, known as Spanglish. We explore the embedding space to discover how they capture the meanings of words in both languages. We test the effectiveness of these embeddings by participating in SemEval 2020 Task 9: Sentiment Analysis on Code-Mixed Social Media Text. We utilised them to train a sentiment classifier that achieves an F-1 score of 0.722. This is higher than the baseline for the competition of 0.656, with our team (codalab username francesita) ranking 14 out of 29 participating teams, beating the baseline. 1 Introduction Sentiment Analysis (SA), or opinion mining, aims to identify subjective information from text. SA plays a pivotal role in promoting and identifying reception of products in platforms such as Amazon, or identifying the sentiment and opinions on numerous topics on social media platforms, such as Twitter or Facebook. As a consequence, SA is a popular task in the Natural Language Processing (NLP) community. Sentiment analysis of social media posts has been a popular research topic, and has naturally led to the sentiment analysis of posts in languages other than English, as well as code-switched posts. Written text in which multiple languages co-exist, known as code-switching or code-mixing, is now more abundant thanks to the ample use of social media. Recent research on code-switched text has focused on using multilingual word embeddings, however, these embeddings were not trained using code-switched text. In this work, we present word-embeddings trained on code-switched social media text, which serves as our main contribution for SemEval 2020 Task 9 1. This task consisted of classifying tweets into one of three classes: positive, neutral, or negative. The dataset used for this task is the one provided by the competition organisers, who collected and annotated the corpus, (Patwa et al., 2020). We trained a BiLSTM (Schuster and Paliwal, 1997) classifier to detect the sentiment of code-mixed tweets by creating our own code-switched embeddings. Using the techniques outlined in section 3, large amounts of code-switched text can be collected to train code-switched embeddings for different NLP applications, such as semantic processing, dependency parsing (Schuster et al., 2019), as well as sentiment analysis, all of which have been identified as challenges posed by code-mixing in NLP tasks (Patwa et al., 2020). We …

Content truncated to 500 words