NLP-CIC at SemEval-2020 Task 9: Analysing Sentiment in Code-switching Language Using a Simple Deep-learning Classifier

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 957–962 Barcelona, Spain (Online), December 12, 2020. 957 NLP-CIC at SemEval-2020 Task 9: Analysing sentiment in code-switching language using a simple deep-learning classifier Jason Angel CIC, Instituto Politécnico Nacional Mexico City, Mexico ajason08@gmail.com Segun Taofeek Aroyehun CIC, Instituto Politécnico Nacional Mexico City, Mexico aroyehun.segun@gmail.com Antonio Tamayo CIC, Instituto Politécnico Nacional Mexico City, Mexico ajtamayoh@gmail.com Alexander Gelbukh CIC, Instituto Politécnico Nacional Mexico City, Mexico www.gelbukh.com Abstract Code-switching is a phenomenon in which two or more languages are used in the same message. Nowadays, it is quite common to find messages with languages mixed in social media. This phenomenon presents a challenge for sentiment analysis. In this paper, we use a standard convolu- tional neural network model to predict the sentiment of tweets in a blend of Spanish and English languages. Our simple approach achieved a F1-score of 0.71 on test set on the competition. We analyze our best model capabilities and perform error analysis to expose important difficulties for classifying sentiment in a code-switching setting. 1 Introduction The phenomenon of combining two or more languages in the same message is known as code-switching or code-mixing (Gumperz, 1982; Myers-Scotton, 1993). Code-switching is an indicator of bilingual competence (Hamers and Blanc, 1999), and it is also motivated by social and cultural factors such as social status, race, age, etc. (Kim, 2006). Although this phenomenon has been studied extensively in linguistics (Pfaff, 1979; Poplack, 1980; Gumperz, 1982; Myers-Scotton, 1993; Milroy and Muysken, 1995; Lipski, 2005; Martínez, 2010; Auer, 2013), it is still challenging for machines to process mixed natural languages. Code-switching is notoriously present on social media posts and chats such as Twitter, Facebook or WhatsApp; consequently making it more difficult to process the sentiment expressed in such contents. In this work, we present a Convolutional Neural Network (CNN) system to predict the sentiment of a given code-mixed tweet. The sentiment labels are either positive, negative, or neutral, and the languages involved are English and Spanish. Our best model utilizes only Spanish word embeddings from tweets (Deriu et al., 2017) and does not require manual feature engineering. 2 Related work Sentiment analysis is a widely studied task in monolingual, multilingual and cross-lingual settings. For instance, monolingual opinion mining in a multilingual context (Boiy and Moens, 2009), multilingual sentiment analysis (Balahur and Turchi, 2014), and cross-lingual polarity detection (Demirtas and Pe- chenizkiy, 2013). Language-independent approaches to sentiment analysis include the use of emoticons (Davies and Ghahramani, 2011) or emoticons and noisy labels (Narr et al., 2012). Sentiment analysis has not been extensively studied on code-switched content. A possible reason is the paucity of large annotated This work is licensed under a Creative Commons Attribution 4.0 International License. License details : http:// creativecommons.org/licenses/by/4.0/. 958 data covering several language pairs or combinations. The Sentimix shared task is an effort to address this problem on the Spanish-English and Hindi-English language pairs for sentiment analysis. Analysing opinions in tweets that blend Spanish and English is a difficult task (Vilares …

Content truncated to 500 words