GunadarmaXBRIN at SemEval-2023 Task 12: Utilization of SVM and AfriBERTa for Monolingual, Multilingual, and Zero-shot Sentiment Analysis in African Languages

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 869–877 July 13-14, 2023 ©2023 Association for Computational Linguistics GunadarmaXBRIN at SemEval-2023 Task 12: Utilization of SVM and AfriBERTa for Monolingual, Multilingual, and Zero-shot Sentiment Analysis in African Languages Novitasari Arlim Gunadarma University Depok, Indonesia novitasariarlim@gmail.com Slamet Riyanto National Research and Innovation Agency Jakarta, Indonesia slam011@brin.go.id Rodiah Rodiah Gunadarma University Depok, Indonesia rodiah@staff.gunadarma.ac.id Al Hafiz Akbar Maulana Siagian National Research and Innovation Agency Jakarta, Indonesia alha001@brin.go.id Abstract This paper describes our participation in Task 12: AfriSenti-SemEval 2023, i.e., Track 12 of subtask A, Track 16 of subtask B, and Track 18 of subtask C. To deal with these three tracks, we utilize Support Vector Machine (SVM) + One vs Rest, SVM + One vs Rest with SMOTE, and AfriBERTa-large models. In particular, our SVM + One vs Rest with SMOTE model could obtain the highest weighted F1-Score for Tracks 16 and 18 in the evaluation phase, that is, 65.14% and 33.49%, respectively. Mean- while, our SVM + One vs Rest model could perform better than other models for Track 12 in the evaluation phase. 1 Introduction Sentiment Analysis (SA) is one of the Natural Lan- guage Processing (NLP) tasks. SA is used to recog- nize and classify the author’s opinion or experience of an object, such as a specific topic, product, ser- vice, organization, social events, politics, and eco- nomic (Nahili et al., 2021; Yang, 2021; Mehmood et al., 2019). The sentiment of the opinion could be classified as positive, neutral, or negative. Im- plementation of SA is employed in various sec- tors, such as business, healthcare, and education (Nandwani and Verma, 2021). However, SA re- search mostly focused on high-resource language, i.e., English and Chinese (Nasim and Ghani, 2020). In other words, low-resource language is still not yet addressed well (Muhammad et al., 2022), such as African language (Muhammad et al., 2023b), which is the focus of task 12. Task 12 AfriSenti-SemEval1 (Muhammad et al., 2023b) aims to do sentiment analysis on African 1https://codalab.lisn.upsaclay.fr/ competitions/7320 datasets collected from Twitter. This task 12 con- sists of three subtasks, i.e., monolingual sentiment classification (subtask A), multilingual sentiment classification (subtask B), and zero-shot sentiment classification (subtask C). In total, there are 14 datasets provided in the task 12, namely, Hausa, Yoruba, Igbo, Nigerian Pidgin, Amharic, Alge- rian Arabic, Moroccan Arabic/Darija, Swahili, Kin- yarwanda, Twi, Mozambican Portuguese, Xitsonga (Mozambique Dialect), Tigrinya, and Oromo. In this AfriSenti-SemEval task, we participate in Track 12: Xitsonga (Mozambique Dialect) of subtask A, Track 16: 12 languages in subtask A of subtask B, and Track 18: Zero-Shot on Oromo of subtask C. To deal with each subtask, we uti- lize three models, namely, Support Vector Machine (SVM) + One vs Rest (Model 1), SVM + One Vs Rest with SMOTE (Model 2), and AfriBERTa- large (Ogueji et al., 2021) (Model 3). The highest- obtained F1-Scores of our participation are 50.56%, 65.14%, and 33.49% for Track 12, Track 16, and Track 18, respectively. The code used in …

Content truncated to 500 words