UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource Languages

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1054–1060 July 13-14, 2023 ©2023 Association for Computational Linguistics UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource languages Egil Rønningstad University of Oslo Department of Informatics egilron@ifi.uio.no Abstract Our contribution to the 2023 AfriSenti- SemEval shared task 12: Sentiment Analysis for African Languages, provides insight into how a multilingual large language model can be a resource for sentiment analysis in languages not seen during pretraining. The shared task provides datasets of a variety of African lan- guages from different language families. The languages are to various degrees related to lan- guages used during pretraining, and the lan- guage data contain various degrees of code- switching. We experiment with both monolin- gual and multilingual datasets for the final fine- tuning, and find that with the provided datasets that contain samples in the thousands, monolin- gual fine-tuning yields the best results. 1 Introduction The 2023 AfriSenti-SemEval Shared Task 12 is the first SemEval shared task for sentiment analysis, targeting African low-resource languages (Muham- mad et al., 2023b). It aims to raise awareness for the need of annotated data in languages that receive little attention when it comes to building AI tools for the digital world. The task is, for each tweet in the dataset, to classify them correctly as conveying a negative, neutral or positive sentiment. This task of classi- fying sentiment category for microblog statements or individual sentences is a useful component in various Natural Language Processing (NLP) tasks. The problem is well researched for English, where similar tasks are modelled with more than 97% accuracy.1 The shared task at hand is split in subtask A, B and C, where subtask A provides training data for 12 African languages, and subtask B provides a joint, multilingual train set for the same languages. 1https://paperswithcode.com/task/ sentiment-analysis We did not participate in subtask C which provided test data in two languages for Zero-shot inference. Our work shares insight to the effect of fine- tuning a multilingual large language model (llm), for languages not seen during pretraining. We com- pare the performance of the resulting model with its exposure to similar languages during the various steps of training and fine-tuning. We find that mod- els fine-tuned on larger training sets, and models fine-tuned on languages close to those seen during pretraining and initial fine-tuning, perform the best. We found the "XLM-Twitter-sentiment" model pre- sented in Section 3 to be the best starting point according to our constraints. This model is both adapted to multilingual Twitter data in pretraining, and is initially fine-tuned to sentiment classification on a multilingual Twitter sentiment classification dataset. We fine-tuned this model on the provided mono- lingual data, and compared this with alternative models fine-tuned on the the multilingual dataset containing all 12 languages, and also models trained on a concatenation of the training data for the languages in the same language family. This work is presented in Section 5. We found that for the …

Content truncated to 500 words