YNU-HPCC at SemEval-2023 Task 9: Pretrained Language Model for Multilingual Tweet Intimacy Analysis

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 733–738 July 13-14, 2023 ©2023 Association for Computational Linguistics YNU-HPCC at SemEval-2023 Task 9: Pretrained Language Model for Multilingual Tweet Intimacy Analysis Qisheng Cai, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Kunming, China Contact: kujou@mail.ynu.edu.cn, {wangjin, xjzhang}@ynu.edu.cn Abstract This paper describes our fine-tuned pretrained language model for task 9 (Multilingual Tweet Intimacy Analysis, MTIA) of the SemEval 2023 competition. MTIA aims to quantitatively analyze tweets in 6 languages for intimacy, giv- ing a score from 1 to 5. The challenge of MTIA is in semantically extracting information from code-mixed texts. To alleviate this difficulty, we suggested a solution that combines attention and memory mechanisms. The preprocessed tweets are input to the XLM-T layer to get sen- tence embeddings and subsequently to the bidi- rectional GRU layer to obtain intimacy ratings. Experimental results show an improvement in the overall performance of our model in both seen and unseen languages. 1 Introduction Intimacy involves lots of social information (e.g., social norms). (Pei and Jurgens, 2020) By evaluat- ing the intimacy score of tweets crawled on social media sites like Twitter quantitatively, it is possi- ble to mine more meaningful social information. MTIA aims to build a sentiment analysis model (or system) to predict the intimacy scores, rang- ing from one to five, of tweets in a Twitter dataset containing ten languages, including English, Span- ish, Portuguese, Italian, French, Chinese, Hindi, Dutch, Korean and Arabic. Among those ten lan- guages, the first six are observable (i.e., appear dur- ing training), and the rest are unobservable (used for zero-shot learning). Tweets abound with online slang, emojis and face characters, which usually contain extra inti- mate thoughts. Thus, it is critical to extract seman- tic information from those unconventional texts. However, many accessible pre-trained models (e.g., XLM) lack consideration of non-traditional text in the training process. The non-traditional text needs elimination to de-noise when we perform transfer learning on these models. Another feasible ap- proach is to convert these non-traditional symbols to conventional words. However, the extraction of information from face characters remains a chal- lenge. Fortunately, we can perform transfer training us- ing the XLM-T model (Barbieri et al., 2021) trained on the Twitter dataset, thus avoiding information loss in the symbolic text. Additionally, we sug- gest a better way to link bidirectional GRU (Cho et al., 2014) layers at the XLM-T. The approach, which combines attention and memory mecha- nisms, is inspired by the analogy to humans. We will demonstrate experimentally that this measure enhances the model’s overall performance, both seen and unseen text. Our model achieved 13/45 places in terms of overall performance in the leader- board, with both Dutch and Arabic ranking fourth. The code will be available at https://github. com/cskujou/tweet-intimacy. The roadmap for this paper is as follows. Section 2 describes the work related to sentiment analysis. Section 3 describes the architecture of our model. Section 5 reports the experimental results. Section …

Content truncated to 500 words