jelenasteam at SemEval-2023 Task 9: Quantification of Intimacy in Multilingual Tweets using Machine Learning Algorithms: A Comparative Study on the MINT Dataset

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 638–643 July 13-14, 2023 ©2023 Association for Computational Linguistics jelenasteam at SemEval-2023 Task 9: Quantification of Intimacy in Multilingual Tweets using Machine Learning Algorithms: A Comparative Study on the MINT Dataset Jelena Lazi´c PhD student School of Electrical Engineering University of Belgrade lazic.jelenaa@gmail.com Sanja Vujnovi´c Assistant Professor School of Electrical Engineering University of Belgrade svujnovic@etf.bg.ac.rs Abstract Intimacy is one of the fundamental aspects of our social life. It relates to intimate interac- tions with others, often including verbal self- disclosure. In this paper, we researched ma- chine learning algorithms for quantification of the intimacy in the tweets. A new multilingual textual intimacy dataset named MINT was used. It contains tweets in 10 languages, including English, Spanish, French, Portuguese, Italian, and Chinese in both training and test datasets, and Dutch, Korean, Hindi, and Arabic in test data only. In the first experiment, linear re- gression models combine with the features and word embedding, and XLM-T deep learning model were compared. In the second experi- ment, cross-lingual learning between languages was tested. In the third experiments, data was clustered using K-means. The results indicate that XLM-T pre-trained embedding might be a good choice for an unsupervised learning algo- rithm for intimacy detection. 1 Introduction Natural language processing (NLP) models have access to more information than any human speaker during their life. Still, it would be hard for them to reach human-level competence and performance (Hovy and Yang, 2021). NLP community needs to address social factors to get closer to the human- like language understanding. Performances of the NLP models for detection of the social-components in the language like patronizing and condescending language detection, irony and sarcasm understand- ing, persuasion, humor and emphasis detection, and many others, should be improved. The aim of our work was analysis of intimacy in tweets, as intimacy is one of the fundamental social aspects of language. In this paper, a new multilingual textual intimacy dataset named MINT annotated by (Pei et al., 2023) was used. The dataset is released along with the SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysis, co-organized by University of Michigan and Snap Inc. It contains a total of 13, 384 tweets in 10 languages, including English, Spanish, French, Portuguese, Italian, and Chinese in both training and test data, and Dutch, Korean, Hindi, and Arabic in test data only. During work on this task, a three experiments were conducted: • Models Comparision Recently, models based on neural networks have become increasingly popular. While these models achieve very good performance in practice, they tend to be relatively slow both at training and test time, limiting their use on very large datasets (Joulin et al., 2016). Meanwhile, traditional models are often con- sidered as strong baselines. In the first ex- periment comparative study of feature-based, embedding-based and deep learning (DL) models was conducted. • Cross-lingual evaluation Zero-shot learning in NLP is still challenging. In their work (Mozafari et al., 2022) show that only …

Content truncated to 500 words