I2C-Huelva at SemEval-2023 Task 9: Analysis of Intimacy in Multilingual Tweets Using Resampling Methods and Transformers

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 758–762 July 13-14, 2023 ©2023 Association for Computational Linguistics Abstract Nowadays, intimacy is a fundamental aspect of how we relate to other people in social settings. The most frequent way in which we can determine a high level of intimacy is in the use of certain emoticons, curse words, verbs, etc. This paper presents the approach developed to solve SemEval 2023 task 9: Multiligual Tweet Intimacy Analysis. To address the task, a transfer- learning approach was conducted by fine- tuning various pre-trained language models. Since the dataset supplied by the organizer was highly imbalanced, our main strategy to obtain high prediction values was the implementation of round-trip translation for oversampling and a random approach for undersampling on the training set. Our final submission achieved an overall Pearson’s r of 0.497. 1 Introduction Intimacy is defined as the set of thoughts and feelings that human beings keep private. There are many ways of expressing a thought and not all are equally intimate, so it is important to know what determines the degree of intimacy of a particular Tweet. To detect this type of language, SemEval 2023 proposed the task 9: Multiligual Tweet Intimacy Analysis (Pei et al., 2023). The aim of the proposed task is to quantify the intimacy expressed in a set of tweets written in six languages: Chinese, English, Spanish, Portuguese, French and Italian. We addressed the problem using regression (Specht et al., 1991) and neural networks (Zhou et al., 2015). We utilized different pre-trained models such as RoBERTa (Liu et al., 2019) or BERT (Devlin et al., 2018), available on the Hugging Face platform. Given the strong imbalance in the data, we have focused on applying different oversampling (Liu et al., 2007) and undersampling (Bach et al., 2019) strategies. The main contributions addressed in this work are as follows: • Analysis of sampling methods to mitigate bias in data. • Benchmarking of different pre-trained multilingual and monolingual models. The rest of this paper is organized as follows: in Section 2 we explain the dataset, the meaning of the labels and data distribution. Therefore, we refer to other studies that contribute to our approach. Section 3 discusses the approaches applied to achieve better model performance. In Section 4, we explain the libraries used. In Section 5, the results obtained in test phase are shown. Finally, conclusions and future works are described in Section 6. 2 Background The training dataset consists of 9491 tweets written in six languages. The distribution of each of the languages is balanced. Figure 1 shows an example of a dataset row. Text Label Language @user was that YOU dancing in ur recent story???!!! 😯 4.0 English Figure 1. Dataset example • The main fields are: Text. Represents the tweet to be analyzed. • Label. Determines the level of intimacy assigned to the text with a range [1-5], where 1 represents the lowest level and 5 represents the highest level of intimacy. • Language. …

Content truncated to 500 words