Sefamerve at SemEval-2023 Task 12: Semantic Evaluation of Rarely Studied Languages

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 512–516 July 13-14, 2023 ©2023 Association for Computational Linguistics Sefamerve at SemEval-2023 Task 12: Semantic Evaluation of Rarely Studied Languages Selman Delil Sefamerve R&D Center Istanbul, Turkey selman.delil@sefamerve.com selmandelil@gmail.com Birol Kuyumcu Sefamerve R&D Center Istanbul, Turkey birol.kuyumcu@sefamerve.com birolkuyumcu@gmail.com Abstract This paper describes our contribution to SemEval-23 Shared Task 12: ArfiSenti. The task consists of several sentiment classification subtasks for rarely studied African languages to predict positive, negative, or neutral classes of a given Twitter dataset. In our system we utilized three different models; FastText, MultiLang Transformers, and Language-Specific Trans- formers to find the best working model for the classification challenge. We experimented with mentioned models and mostly reached the best prediction scores using the Language Specific Transformers. Our best-submitted result was ranked 3rd among submissions for the Amharic language, obtaining an F1 score of 0.702 be- hind the second-ranked system. 1 Introduction Sentiment analysis is a task in natural language processing (NLP) that involves identifying and ex- tracting the sentiment expressed in a text. It has many practical applications, such as in business and marketing analysis, product review analysis, and social media monitoring (Hutto and Gilbert, 2014). While there has been considerable research in senti- ment analysis for widely spoken languages such as English, there has been relatively little work done for African languages. This is mainly due to the scarcity of labeled data and language resources for African languages, which hinders the development of effective sentiment analysis models for these languages (Muhammad et al., 2023a). The AfriSenti task (Muhammad et al., 2023b,a) motivated the need for curated datasets for African languages to enable various AI applications such as sentiment analysis, machine translation, and hate- ful content detection. Due to the lack of such datasets for most African languages, the impor- tance of research to determine the suitability of current NLP techniques and the development of novel techniques for these languages are critical. The task aims to evaluate sentiment analysis methods for African languages, specifically for Twitter data. The AfriSenti-SemEval Shared Task 12 involves sentiment classification of tweets in 14 African languages, with three sub-tasks includ- ing monolingual sentiment classification, multilin- gual sentiment classification, and zero-shot sen- timent classification. Participants can select one or more sub-tasks and participate in any number of languages. Task A has 15 tracks with training data in different languages, while Task B has only one track with 12 languages. Task C involves de- termining the sentiment of unlabelled tweets in two African languages using any available training datasets (Muhammad et al., 2023a, 2022). In this paper, we describe our approach to the AfriSenti-SemEval Shared Task 12, where we ex- perimented with three different models to find the best working model for sentiment classification in African languages. We adopted the FastText em- bedding model, the AfriBERTa transformer-based language model, and language-specific transformer models. Through our experimentation, we aimed to determine the most effective model for sentiment classification on the provided dataset. Our best results …

Content truncated to 500 words