ROZAM at SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysis

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2029–2032 July 13-14, 2023 ©2023 Association for Computational Linguistics ROZAM at SemEval-2023 Task 9: Multilingual Tweet Intimacy Analysis Mohammadmostafa Rostamkhani, Ghazal Zamaninejad, Sauleh Eetemadi Iran University of Science and Technology {mo_rostamkhani97, gh_zamaninejad@comp.iust.ac.ir, sauleh@iust.ac.ir Abstract We build a model using large multilingual pre- trained language model XLM-T for regression task and fine-tune it on the MINT (Multilin- gual INTmacy) analysis dataset which covers 6 languages for training and 4 languages for test- ing zero-shot performance of the model. The dataset was annotated and the annotations are intimacy scores. We experiment with several deep learning architectures to predict intimacy score. To achieve optimal performance we modify several model settings including loss function, number and type of layers. In total, we ran 16 end-to-end experiments. Our best system achieved a Pearson Correlation score of 0.52. 1 Introduction Intimacy has long been considered as the main dimension of human relationships (Maslow, 1981; Sullivan, 2013; Prager, 1995). Although intimacy has a significant role in lan- guage, there are few datasets in this field. The first textual intimacy dataset (Pei et al., 2023) contains 2,397 English questions collected through social media. However, models trained on this dataset may not generalize well to other situations. MINT dataset (Pei et al., 2022) is a textual inti- macy dataset covering 10 languages which is col- lected from people’s tweets and annotated with an intimacy score of 1 to 5. The task is to predict intimacy scores of tweets for 10 languages. Six lan- guages are seen during training including English, Spanish, Portuguese, Italian, French, Chinese and 4 other languages (Hindi, Korean, Dutch, and Ara- bic) are not included in training data. These four languages are used for evaluating zero-shot perfor- mance of proposed models. By doing this task, we can extract some social information inside the sentence and hence calculate how intimate it is. XLM-RoBERTa (Conneau et al., 2019) is a mul- tilingual pre-trained language model, pre-trained on 2.5TB of filtered CommonCrawl data contain- ing 100 languages (it is a multilingual version of RoBERTa) and it was pre-trained with the Masked Language Modeling (MLM) objective. The main strategy is to use XLM-T (Barbieri et al., 2021) which is a XLM-Roberta-base model trained on about 198M multilingual tweets. Our model contains XLM-T for extracting em- beddings of sentences and also some dense and dropout layers for predicting (approximating) the intimacy score of a sentence. We fine-tune the last few layers of XLM-T and train the regressor part of our model using the training dataset. Since our dataset was relatively small, we also used some data augmentation methods. Eventually, teams are ranked based on Pearson Correlation metric cal- culated on overall score for seen and unseen lan- guages. The proposed method was ranked 33 out of 45. The details of our implementations are avail- able through our github repository. 2 Background The inputs to our model were tweets collected from twitter and we expect our model to produce a …

Content truncated to 500 words