TueMix at SemEval-2020 Task 9: Logistic Regression with Linguistic Feature Set

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1316–1321 Barcelona, Spain (Online), December 12, 2020. 1316 TueMix at SemEval-2020 Task 9: Logistic Regression with Linguistic Feature Set for Sentiment Analysis of Code-Mixed Social Media Text Elizabeth Bear Diana Constantina H¨ofels Mihai Manolescu {elizabeth.bear, diana-constantina.hoefels, mihai.manolescu} @student.uni-tuebingen.de Eberhard-Karls University of T¨ubingen Abstract Commonly occurring in settings such as social media platforms, code-mixed content makes the task of identifying sentiment notably more challenging and complex due to the lack of structure and noise present in the data. SemEval-2020 Task 9, SentiMix, was organized with the purpose of detecting the sentiment of a given code-mixed tweet comprising Hindi and English. We tackled this task by comparing the performance of a system, TueMix - a logistic regression algorithm trained with three feature components: TF-IDF n-grams, monolingual sentiment lexicons, and surface features - with a neural network approach. Our results showed that TueMix outperformed the neural network systems and the addition of the linguistic features beyond TF-IDF n-grams enhanced our performance, yielding a weighted F1-score of 0.685. 1 Introduction The task of sentiment analysis with code-mixing is gaining more and more attention, given that on social media platforms, bilingual or multilingual individuals create content by mixing two or more different languages within a single sentence. Code-mixing occurs when phrases, words, and morphemes of one language are inserted into a sentence of another language (Sridhar and Sridhar, 1980). People may code-mix to make themselves appear noteworthy or imply a certain social status (Shbat, 2007), to show solidarity, or simply to express their feelings (Holmes, 2008). A substantial number of individuals combine English and Hindi on social media platforms, and as a result, the need to process code-mixed content has become essential if we want to build robust and competitive systems for sentiment detection. Sentiment analysis of code-mixed text faces multiple challenges due to the lack of a formal grammar for the hybrid code-mixed generated language and the existing characteristics of multilingual content where there are atypical variations in spelling, ungrammatical structures, and transliteration (Bali et al., 2014). The International Workshop on Semantic Evaluation, or SemEval, addressed this challenge by proposing the shared task SentiMix: Sentiment Analysis for Code-Mixed Social Media Text (Patwa et al., 2020). The training data provided contained 14000 Hindi-English tweets annotated with the corresponding sentiment label (positive, neutral, or negative), and with additional language labels at the word-level (English (Eng), Hindi (Hin), emoji (EMT), or universal (0) for other tokens). The class distribution was fairly balanced with 33.1% positive, 37.6% neutral, and 29.29% negative tweets. Our strategy consisted of a machine learning and a neural network-based approach. Our main system, TueMix, used a logistic regression algorithm trained with three categories of features: TF-IDF n-grams, monolingual sentiment lexicons, and a number of surface features. In our findings, TueMix achieved better results than the neural network systems, placing in the top third of 62 teams (listed as “guzimanis” in the official CodaLab rankings). The addition of the linguistic features beyond the TF-IDF n-grams …

Content truncated to 500 words