IIITG-ADBU at SemEval-2020 Task 9: SVM for Sentiment Analysis of English-Hindi Code-Mixed Text

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 946–950 Barcelona, Spain (Online), December 12, 2020. 946 IIITG-ADBU at SemEval-2020 Task 9: SVM for Sentiment Analysis of English-Hindi Code-Mixed Text Arup Baruah1, Kaushik Amar Das1, Ferdous Ahmed Barbhuiya1, and Kuntal Dey2∗ 1IIIT Guwahati, India 2Accenture Technology Labs, Bangalore arup.baruah@gmail.com, kaushikamardas@gmail.com, ferdous@iiitg.ac.in, kuntal.dey@accenture.com Abstract In this paper, we present the results that the team IIITG-ADBU (codalab username ‘abaruah’) obtained in the SentiMix task (Task 9) of the International Workshop on Semantic Evaluation 2020 (SemEval 2020). This task required the detection of sentiment in code-mixed Hindi-English tweets. Broadly, we performed two sets of experiments for this task. The first experiment was performed using the multilingual BERT classifier and the second set of experiments was performed using SVM classifiers. The character-based SVM classifier obtained the best F1 score of 0.678 in the test set with a rank of 21 among 62 participants. The performance of the multilingual BERT classifier was quite comparable with the SVM classifier on the development set. However, on the test set it obtained an F1 score of 0.342. 1 Introduction Sentiment analysis has been defined as the computational study of opinions, sentiments, and emotions expressed in the text (Liu, 2010). In its basic form, sentiment analysis is used to determine the polarity of a given text where the polarity may be negative, neutral, and positive. Thus, sentiment analysis can be viewed as a text classification problem. With the advent of social media platforms, the use of non-standard language has increased. Now- a-days, it is very common to use emoticons, mentions, acronyms, and ungrammatical sentences while communicating in social media. All these factors make the traditional tools used for natural language processing fail on social media text. Another new style of communication in social media is the use of code-mixed text. Code-mixing means the mixing of words from more than one language in the same sentence or between sentences. Code mixing makes the task of sentiment analysis more challenging. The objective of SentiMix (Task 9), organized as part of the International Workshop on Semantic Evaluation 2020 (SemEval 2020), is to detect the sentiment of code-mixed tweets (Patwa et al., 2020). This task was a three-way classification problem with the labels being negative, neutral, and positive. The task was held for both Hindi-English and Spanish-English code-mixed tweets. We participated in this task for the Hindi-English language. In this task, we experimented with SVM and multilingual BERT classifiers. 2 Related Work Joshi et al. (2010) performed the first work on the detection of sentiment analysis of Hindi text. In their work, unigram and bigram based SVM classifiers were used. As another approach, the Hindi text was machine translated to English and the translated text was then classified using a classifier trained on English text. This work also led to the creation of Hindi-SentiWordNet sentiment lexicon. The lexicon was also used to perform a lexicon-based sentiment analysis. The classifier trained on the Hindi text performed the best with an accuracy of 78.14%. …

Content truncated to 500 words