ODA_SRIB at SemEval-2023 Task 9: A Multimodal Approach for Improved Intimacy Analysis

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1676–1680 July 13-14, 2023 ©2023 Association for Computational Linguistics ODA_SRIB at SemEval-2023 Task 9: A Multimodal Approach for Improved Intimacy Analysis Priyanshu Kumar, Amit Kumar, Jiban Prakash, Prabhat Lamba and Irfan Abdul {priyanshu.k, amit.kumar, p.jiban, prabhat.l, irfan.abdul}@samsung.com Samsung Research and Development Institute Bangalore, India Abstract We experiment with XLM-Twitter and XLM- RoBERTa models to predict the intimacy scores in Tweets i.e. the extent to which a Tweet contains intimate content. We propose a Transformer-TabNet based multimodal archi- tecture using text data and statistical features from the text, which performs better than the vanilla Transformer based model. We further experiment with Adversarial Weight Perturba- tion to make our models generalized and ro- bust. The ensemble of four of our best mod- els achieve an over-all Pearson Coefficient of 0.5893 on the test dataset. 1 Introduction Intimacy is an important aspect in our society, which in general is not given much attention. With the increasing usage of social media across all age groups, users might post some private intimate in- formation unintentionally. Thus, there is a dire need to create Natural Language Processing (NLP) models to detect the intimate nature of such posts as the user can be nudged before they share some intimate information on social media. Moreover, the capability of understanding intimacy will also help us incorporate emotions in voice assistants and chatbots. For the Task 9 of SemEval 2023: Multi- lingual Tweet Intimacy Analysis (Pei et al., 2022), the organizers intend the participants to build mod- els for detecting the intimacy of a given Tweet on a scale of 1 to 5. They provide a dataset of anno- tated Tweets in prominent languages used around the world - English, Spanish, Portuguese, Italian, French and Chinese. In order to validate the gener- alization of submitted systems, they also have test samples in Hindi, Dutch, Korean and Arabic. In this paper, we present our methodology to develop such models. We formulated the problem as a binary classification problem on soft labels. We rely on XLM-Twitter (Barbieri et al., 2021) and XLM-Roberta (Conneau et al., 2019) models’ cross lingual transfer capability for unseen languages. We devise a multi-modal architecture with text and handcrafted features as input. We experimented with augmented data which we created by translat- ing the text while keeping the intimacy score the same. We incorporate Adversarial Weight Pertur- bation (AWP) (Wu et al., 2020) to train our models in a robust manner. Our final submission is a mean ensemble of four of our best models. Pearson’s coefficient is used to compare the mod- els submitted to the task leaderboard. Our submis- sion attains an overall score of 0.5893 with 0.7318 on seen languages and 0.411 on unseen languages. 2 Background The shared task of Multilingual Tweet Intimacy Analysis is a text regression problem i.e. for a given a Tweet, we need to predict the intimacy score for it. The organizer provide a dataset comprising of 13,372 tweets …

Content truncated to 500 words