Witcherses at SemEval-2023 Task 12: Ensemble Learning for African Sentiment Analysis

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 357–364 July 13-14, 2023 ©2023 Association for Computational Linguistics Witcherses at SemEval-2023 Task 12: Ensemble Learning for African Sentiment Analysis Monil Gokani K V Aditya Srivatsa Language Technologies Research Center (LTRC) Kohli Center on Intelligent Systems International Institute of Information Technology, Hyderabad {monil.gokani, k.v.aditya}@research.iiit.ac.in radhika.mamidi@iiit.ac.in Radhika Mamidi Abstract This paper describes our system submission for SemEval-2023 Task 12 AfriSenti-SemEval: Sentiment Analysis for African Languages. We propose an XGBoost-based ensemble model trained on emoticon frequency-based features and the predictions of several statistical models such as SVMs, Logistic Regression, Random Forests, and BERT-based pre-trained language models such as AfriBERTa and AfroXLMR. We also report results from additional experi- ments not in the system. Our system achieves a mixed bag of results, achieving a best rank of 7th in three of the languages - Igbo, Twi, and Yoruba. 1 Introduction The AfriSenti Shared Task (Muhammad et al., 2023b) is aimed at promoting sentiment classifi- cation research in a diverse group of African Lan- guages. Though sentiment classification is an ex- tremely popular task in NLP, there has been rela- tively little work done for it in African Languages. Apart from the NaijaSenti dataset (Muhammad et al., 2022) (which is a part of the task itself), some of the languages where such work has been done include Amharic (Yimam et al., 2020), Tunisian Arabizi (Fourati et al., 2021), and Swahili (Martin et al., 2021). The task itself is divided into three sub-tasks - monolingual classification in 12 lan- guages separately, multilingual classification, and zero-shot classification for two languages. We participated in the monolingual and mul- tilingual tracks of the task. Our system con- sists of an ensemble model that leverages emoti- con frequencies and the predictions of contextu- alised, transformer-based models such as AfriB- ERTa (Ogueji et al., 2021) and AfroXLMR (Al- abi et al., 2022), as well as simpler models such as Logistic Regression, SVM and Random Forest classifiers. We also release the code on GitHub.1 1https://github.com/MG1800/afrisenti-ensemble Our model achieves a significant range of ranks, with our best result being ranked 7th each in the Igbo, Twi, and Yoruba tracks and between 12 and 15 on the multilingual, Hausa and Xitsonga tracks. The full rankings are reported in Table 2. We report observations on the performances of our individual models that we used across lan- guages and the performance of languages across models. Additionally, we report the results for some of the experiments we carried out that did not make it to the final model. 2 Background 2.1 Task Description The objective of the task (Muhammad et al., 2023b) is to identify the polarity of a tweet (negative, posi- tive, or neutral) in a set of 14 African languages. It is divided into three sub-tasks. Sub-Task A is for monolingual classification systems, where a separate classifier is trained for each language and has a separate track for each of the 12 languages in this sub-task. These lan- guages are Hausa, Yoruba, …

Content truncated to 500 words