DUTH at SemEval-2023 Task 9: An Ensemble Approach for Twitter Intimacy Analysis

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1225–1230 July 13-14, 2023 ©2023 Association for Computational Linguistics DUTH at SemEval-2023 Task 9: An Ensemble Approach for Twitter Intimacy Analysis Georgios Arampatzis Vasileios Perifanis Symeon Symeonidis Avi Arampatzis Database & Information Retrieval research unit, Department of Electrical & Computer Engineering, Democritus University of Thrace {geoaramp,vperifan,ssymeoni,avi}@ee.duth.gr Abstract This work presents the approach developed by the DUTH team for participating in the SemEval-2023 Task 9: Multilingual Tweet In- timacy Analysis. Our results show that pre- processing techniques do not affect the learn- ing performance for the task of multilingual intimacy analysis. In addition, we show that fine-tuning a transformer-based model does not provide advantages over using the pre-trained model to generate text embeddings and using the resulting representations to train simpler and more efficient models such as Multilayer Perceptron (MLP). Finally, we utilize an ensem- ble of classifiers, including three MLPs with different architectures and a CatBoost model, to improve the regression accuracy. 1 Introduction Intimacy analysis is concerned with identifying and analyzing the level of emotional closeness between individuals using natural language (Pei and Jurgens, 2020). This form of analysis has a variety of appli- cations in domains such as social media, healthcare and marketing, providing valuable insights into user behaviour, sentiment and preferences (Prager, 2005). Despite the significance of intimacy as a reflec- tion of social norms (Pei and Jurgens, 2020; Prager, 2005; Pei et al., 2022), there is limited availabil- ity of resources for analyzing it using machine learning techniques. To address this issue, Pei et al. (2022) processed and prepared a multilin- gual dataset consisting of tweets in ten languages, including English, Spanish, French, Portuguese, Italian, Chinese, Hindi, Korean, Dutch, and Arabic. This dataset has been made publicly available to evaluate the performance of computational tools that analyse intimacy. This paper presents an ensemble intimacy regres- sion scheme based on an extensive experimental study of several regressor models (Breiman, 2001; Chen and Guestrin, 2016; Prokhorenkova et al., 2018; Haykin, 1994) and text-to-feature represen- tations (Ramos et al., 2003; Le and Mikolov, 2014; Conneau et al., 2020; Barbieri et al., 2022). Our study shows that training simple models using text embeddings generated by transformer-based mod- els leads to effective and rapid training and accurate results. Our models achieve state-of-the-art per- formance without fine-tuning a huge and training- costly transformer-based model. We carefully opti- mized the components of our final ensemble, con- sidering the influence of the imbalanced regression task. Our team achieved a rank of 7 in the overall score, 25 using the languages seen during training, and 2 on cross-language transfer learning. The remainder of this work is structured as fol- lows. The proposed system is described in Sec- tion 2. Section 3 reports on our experiments. Fi- nally, conclusions and future directions are dis- cussed in Section 4. 2 System Description The primary objective of SemEval-2023 Task 9 was to assign a continuous value within the range of [1, 5] to multilingual tweet texts, …

Content truncated to 500 words