C1 at SemEval-2020 Task 9: SentiMix: Sentiment Analysis for Code-Mixed Social Media Text Using Feature Engineering

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1227–1232 Barcelona, Spain (Online), December 12, 2020. 1227 C1 at SemEval-2020 Task 9: SentiMix: Sentiment Analysis for Code-Mixed Social Media Text using Feature Engineering Laksh Advani and Clement Lu and Suraj Maharjan Capital One 1600 Capital One Drive, McLean, VA firstname.lastname@capitalone.com Abstract In today’s interconnected and multilingual world, code-mixing of languages on social media is a common occurrence. While many Natural Language Processing (NLP) tasks like sentiment analysis are mature and well designed for monolingual text, techniques to apply these tasks to code-mixed text still warrant exploration. This paper describes our feature engineering approach to sentiment analysis in code-mixed social media text for SemEval-2020 Task 9: SentiMix. We tackle this problem by leveraging a set of hand-engineered lexical, sentiment, and metadata fea- tures to design a classifier that can disambiguate between “positive”, “negative” and “neutral” sentiment. With this model we are able to obtain a weighted F1 score of 0.65 for the “Hinglish” task and 0.63 for the “Spanglish” tasks. 1 Introduction Social media has grown exponentially in the last decade and has given rise to multilingual online com- munication and the consumption of media. Code Mixing is the phenomenon of embedding linguistic units such as phrases, words, or morphemes of one language into an utterance of another. Code mixing is prevalent in online discourse, where bilingual speakers mix English with their native language. For instance, bilingual speakers of Hindi and Spanish languages, which are the 3rd and 4th most spoken languages in the world respectively, frequently mix English with Hindi and Spanish in spoken language as well as online social media. The growing presence of “Hinglish” and “Spanglish” in social media is evidence that this is a growing area of research as traditional NLP tasks like sentiment analysis heavily rely on monolingual resources. While there are many cutting edge techniques to apply common NLP tasks to monolingual text, the task of sentiment analysis, in particular, has not been explored for multilingual code-mixed texts. Con- ventional NLP tools make use of monolingual resources to operate on code-mixed text, which limits them to properly handle issues like word-level code-mixing. Additionally, code-mixed texts typically exist on social media which has a conjunction of other features like incorrect spelling, use of slang, and abbreviations to name a few. While code-mixing is an active area of research, correctly annotated data is still scarce. It is par- ticularly difficult to mine a small subset of tweets that may or may not be code-mixed and then apply a sentiment score to them. The organizers have solved this by releasing around 20,000 tweets (Patwa et al., 2020) with unique sentiment scores and word-level tagging of the language. For evaluation and ranking, the organizers used a weighted F1 score across “positive”, “negative” and “neutral” classes. While deep learning techniques are state of the art for this task, the contribution of this paper is to demonstrate how content-rich features, when applied to a classifier, can give us strong and …

Content truncated to 500 words