tmn at SemEval-2023 Task 9: Multilingual Tweet Intimacy Detection Using XLM-T, Google Translate, and Ensemble Learning

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1317–1323 July 13-14, 2023 ©2023 Association for Computational Linguistics tmn at SemEval-2023 Task 9: Multilingual Tweet Intimacy Detection using XLM-T, Google Translate, and Ensemble Learning Anna Glazkova University of Tyumen a.v.glazkova@utmn.ru Abstract The paper describes a transformer-based sys- tem designed for SemEval-2023 Task 9: Mul- tilingual Tweet Intimacy Analysis. The pur- pose of the task was to predict the intimacy of tweets in a range from 1 (not intimate at all) to 5 (very intimate). The official training set for the competition consisted of tweets in six lan- guages (English, Spanish, Italian, Portuguese, French, and Chinese). The test set included the given six languages as well as external data with four languages not presented in the train- ing set (Hindi, Arabic, Dutch, and Korean). We presented a solution based on an ensem- ble of XLM-T, a multilingual RoBERTa model adapted to the Twitter domain. To improve the performance of unseen languages, each tweet was supplemented by its English translation. We explored the effectiveness of translated data for the languages seen in fine-tuning compared to unseen languages and estimated strategies for using translated data in transformer-based models. Our solution ranked 4th on the leader- board while achieving an overall Pearson’s r of 0.599 over the test set. The proposed system improves up to 0.088 Pearson’s r over a score averaged across all 45 submissions. 1 Introduction Intimacy is a significant social aspect of language, which helps to explore existing social norms in var- ious contexts (Pei and Jurgens, 2020). The concept of intimacy describes how an individual relates to his addressee in their perceived interdependence, warmth, and willingness to personally share (Perl- man and Fehr, 1987). An automatic evaluation of intimacy in language provides us with a clearer pic- ture of social interactions and different linguistic strategies. This paper describes a system developed for the SemEval-2023 Task 9: Multilingual Tweet Inti- macy Analysis (Pei et al., 2023). The purpose of the task was to predict the intimacy of tweets. The Figure 1: High-level overview of our approach. test set contained tweets in the languages presented (seen) and not presented (unseen) in the training set. The main challenge of the task was the zero- shot intimacy prediction performance for unseen languages. Inspired by the recent success of transformer- based models in multilingual tasks (Malmasi et al., 2022; Tayyar Madabushi et al., 2022), we evaluate the performance of BERT (Bidirectional Encoder Representations from Transformers) (Devlin et al., 2019) and its modifications. In our experiments, we investigated the impact of translation to improve the zero-shot performance. We have found that the joint use of original and translated texts increases the model performance for unseen languages in comparison with the separate use of original or translated data. The paper is organized as follows. Section 2 presents the task and the dataset provided by the task organizers. Our methods are presented in Sec- tion 3. In Section 4, we describe the experimental setups that …

Content truncated to 500 words