SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2319–2337 July 13-14, 2023 ©2023 Association for Computational Linguistics SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval) Shamsuddeen Hassan Muhammad1,2+∗, Idris Abdulmumin3+∗, Seid Muhie Yimam4∗, David Ifeoluwa Adelani5∗, Ibrahim Said Ahmad2+∗, Nedjma Ousidhoum6, Abinew Ali Ayele4,7, Saif M. Mohammad8, Meriem Beloucif9 , Sebastian Ruder10 1University of Porto, 2Bayero University Kano, 3Ahmadu Bello University, Zaria, 4Universität Hamburg, 5University College London, 6University of Cambridge, 7Bahir Dar University, 8National Research Council Canada, 9Uppsala University, 10Google Research, ∗Masakhane NLP, +HausaNLP shmuhammad.csc@buk.edu.ng Abstract We present the first Africentric SemEval Shared task, Sentiment Analysis for African Languages (AfriSenti-SemEval)1. AfriSenti- SemEval is a sentiment classification challenge in 14 African languages (Amharic, Algerian Arabic, Hausa, Igbo, Kinyarwanda, Moroc- can Arabic, Mozambican Portuguese, Nigerian Pidgin, Oromo, Swahili, Tigrinya, Twi, Xit- songa, and Yorùbá) (Muhammad et al., 2023), using data labeled with 3 sentiment classes. We present three subtasks: (1) Task A: monolin- gual classification, which received 44 submis- sions; (2) Task B: multilingual classification, which received 32 submissions; and (3) Task C: zero-shot classification, which received 34 sub- missions. The best performance for tasks A and B was achieved by NLNDE team with 71.31 and 75.06 weighted F1, respectively. UCAS- IIE-NLP achieved the best average score for task C with 58.15 weighted F1. We describe the various approaches adopted by the top 10 systems and their approaches. 1 Introduction Sentiment Analysis is a prominent sub-field of Nat- ural Language Processing that focuses on the au- tomatic identification of sentiments or opinions expressed through online content, such as social media posts, blogs, or reviews (Liu, 2020; Moham- mad, 2021; Nakov et al., 2016). Example applica- tions are the computational analysis of emotions in language, which has been applied to literary analy- sis and culturonomics (Mohammad, 2011; Reagan et al., 2016; Hamilton et al., 2016); commercial use (e.g., tracking opinions towards products); and research in psychology and social science (Dodds et al., 2015; Mohammad et al., 2016). Despite tremendous amount of work in sentiment analysis 1The dataset is available at https://github.com/ afrisenti-semeval/afrisent-semeval-2023. Task A (Monolingual Classification) Track 1: Hausa Track 2: Yoruba Track 3: Igbo Track 4: Nigerian Pidgin Track 5: Amharic Track 6: Algerian Arabic Track 7: Moroccan Arabic/Darija Track 8: Swahili Track 9: Kinyarwanda Track 10: Twi Track 11: Mozambique Portuguese Track 12: Xitsonga (Mozambican) Track 16: Datasets from Track 1-12 Track 17: Tigrinya Track 18: Oromo Task B (Multilingual Classification) Task C (Zero-shot Classification) AfriSenti SemEval 2023 Tasks Figure 1: The AfriSenti-SemEval 2023 Shared Task tracks. over the last two decades, little work has been con- ducted on under-represented languages in general and African languages in particular. Africa has a long and rich linguistic history, ex- periencing language contact, language expansion, development of trade languages, language shift, and language death, on several occasions. The con- tinent is incredibly linguistically diverse and home to over 2000 languages. This includes 75 languages with at least one million speakers each. Africa has a rich tradition of storytelling, poems, …

Content truncated to 500 words