KINLP at SemEval-2023 Task 12: Kinyarwanda Tweet Sentiment Analysis

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 718–723 July 13-14, 2023 ©2023 Association for Computational Linguistics KINLP at SemEval-2023 Task 12: Kinyarwanda Tweet Sentiment Analysis Antoine Nzeyimana* University of Massachusetts, Amherst, MA 01003, USA nzeyi@kinlp.com Abstract This paper describes the system entered by the author to the SemEval-2023 Task 12: Senti- ment analysis for African languages. The sys- tem focuses on the Kinyarwanda language and uses a language-specific model. Kinyarwanda morphology is modeled in a two tier trans- former architecture and the transformer model is pre-trained on a large text corpus using multi-task masked morphology prediction. The model is deployed on an experimental platform that allows users to experiment with the pre- trained language model fine-tuning without the need to write machine learning code. Our final submission to the shared task achieves second ranking out of 34 teams in the competition, achieving 72.50% weighted F1 score. Our anal- ysis of the evaluation results highlights chal- lenges in achieving high accuracy on the task and identifies areas for improvement. 1 Introduction Over the past decade, Twitter has become a ma- jor social media platform with many users among the Kinyarwanda-speaking communities of Eastern and Central Africa. It has become a convenient tool for self-expression, public participation and infor- mation sharing with an impact on the local social, political and cultural environment. This makes it important to study Twitter user content (or tweets) produced by these communities in order to have a well grounded understanding of their sociocultural environment and dynamics. Therefore, performing sentiment analysis on Twitter data can enable appli- cations in different domains such as social studies, public health, business and marketing, governance, art and literature. * This work is part of an independent research and devel- opment effort towards Kinyarwanda language technology. The objective of sentiment analysis for tweets is to uncover the subjective opinion of tweet authors. This requires predicting the author’s sentiment po- larity, which may be positive, negative or neutral. This can be achieved using natural language pro- cessing (NLP) tools. With the recent develop- ments in deep learning methods for NLP (Gold- berg, 2017), especially the use of pre-trained lan- guage models (PLMs) (Devlin et al., 2019; Liu et al., 2019), there is an opportunity to apply these techniques to the tweet sentiment analysis task. However, performing sentiment analysis on tweets is inherently challenging due to multiple factors. First, many Twitter and other social media users use non-standard language in many cases, of- ten using newer symbols such as emojis, handles and hashtags. This makes it harder for traditional parsing tools to handle these new symbols of mean- ing and emotion. Second, in many African lan- guage communities, such as Kinyarwanda speakers, there is often a tendency to code-mixing, whereby users employ words from multiple languages in the same sentence. Third, the training datasets are typically small, making it hard to fit a model to a large and diverse range of topics and language styles. Finally, sentiment polarity can also …

Content truncated to 500 words