IUST at SemEval-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text Using Deep Neural Networks and Linear Baselines

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1270–1275 Barcelona, Spain (Online), December 12, 2020. 1270 IUST at SemEval-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text using Deep Neural Networks and Linear Baselines Soroush Javdan, Taha Shangipour ataei and Behrouz Minaei-Bidgoli Computer Engineering Department Iran University of Science and Technology Tehran, Iran soroush javdan,taha shangipour@comp.iust.ac.ir b minaei@iust.ac.ir Abstract Sentiment Analysis is a well-studied field of Natural Language Processing. However, the rapid growth of social media and noisy content within them poses significant challenges in addressing this problem with well-established methods and tools. One of these challenges is code-mixing, which means using different languages to convey thoughts in social media texts. Our group, with the name of IUST(username: TAHA), participated at the SemEval-2020 shared task 9 on Sentiment Analysis for Code-Mixed Social Media Text, and we have attempted to develop a system to predict the sentiment of a given code-mixed tweet. We used different preprocessing techniques and proposed to use different methods that vary from NBSVM to more complicated deep neural network models. Our best performing method obtains an F1 score of 0.751 for the Spanish-English sub-task and 0.706 over the Hindi-English sub-task. 1 Introduction Sentiment Analysis identification is a sub-field of natural language processing that explores the automatic inference for fine-grained opinion polarity of textual data. The recent growth of social media, review forums and text messaging platforms create a surge in the amount of user-generated textual data, and so increased the urgent need for automatic opinion extraction. However this evolution created many opportunities for language technology and researchers, it provided verity of new challenges, namely, spelling errors, creative invented spelling (”iz” for ”is”), abbreviation (”LOL” for ”laugh out loud”), Meta tags(Hashtags) and code-mixing (Das and Gamb¨ack, 2014). Non-English speakers frequently use multiple languages to express their feelings, which they know as code-mixing. Speakers or writers tend to shift from one language to another either to express their feelings adequately, to show like-mindedness with a group, to distinguish oneself, to discuss a specific topic, or to look impressive to their audience. The SemEval-2020 shared task 9 (Patwa et al., 2020), which is part of the SemEval-2020 workshop, focused on coping with this challenge. This task focus is automatic sentiment analysis in a code-mixed social media text. The task consists of two subtasks, Spanish-English(Spanglish) and Hindi-English(Hinglish) code-mixing, with over 30 groups who participate in each sub-task. This article presents a system that we have implemented for predicting the sentiment of a given code- mixed tweet. The system was developed for both subtasks. Table 1 shows the results that we achieved with our system in the SemEval-2020 competitions. To create a highly accurate classifier, we tested different methods that varied from linear (NBSVM with n-grams) and different deep learning architectures (LSTM, BiLSTM, and combinations). Also, we tested a new architecture based on Wang’s (2019) work on offensive language detection, who used four Convolutional Neural Networks(CNN) with different window sizes(1,2,3,4) and k max-pooling ahead of them. However, the combination …

Content truncated to 500 words