SinaAI at SemEval-2023 Task 3: A Multilingual Transformer Language Model-based Approach for the Detection of News Genre, Framing and Persuasion Techniques

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2168–2173 July 13-14, 2023 ©2023 Association for Computational Linguistics SinaAI at SemEval-2023 Task 3: A Multilingual Transformer Language Model-based Approach for the Detection of News Genre, Framing and Persuasion Techniques Aryan Sadeghi⋄, Reza Alipour⋄, Kamyar Taeb⋄, Parimehr Morassafar⋄, Nima Salemahim⋄, Ehsaneddin Asgari§ ⋄ DH-NLP Lab, Computer Engineering Department, Sharif University of Technology § AI Innovation Center, Data:Lab, Volkswagen AG, Munich, Germany Abstract This paper describes SinaAI’s participation in SemEval-2023 Task 3, which involves detect- ing propaganda in news articles across multi- ple languages. The task comprises three sub- tasks: (i) genre detection, (ii) news framing, and (iii) persuasion technique identification. The employed dataset includes news articles in nine languages and domains, including En- glish, French, Italian, German, Polish, Russian, Georgian, Greek, and Spanish, with labeled in- stances of news framing, genre, and persuasion techniques. Our approach combines fine-tuning multilingual language models such as XLM, LaBSE, and mBERT with data augmentation techniques. Our experimental results show that XLM outperforms other models in terms of F1-Micro in and F1-Macro, and the ensemble of XLM and LaBSE achieved the best perfor- mance. Our study highlights the effectiveness of multilingual sentence embedding models in multilingual propaganda detection. Our mod- els achieved highest score for two languages (Greek and Italian) in sub-task 1 and one lan- guage (Russian) for sub-task 2. 1 Introduction SemEval 2023 Task 3 aims to evaluate the ability of computational models to detect three important aspects of news articles: genre, news framing, and persuasion techniques. The provided dataset in- cludes news articles from various sources, such as newspapers and online media, which have been an- notated with labels indicating the framing, genre, and persuasion technique. Participants are required to submit models that can accurately identify these labels for each article. This task is significant as it provides a means to automatically analyze and clas- sify news articles, which can be useful in various applications such as content filtering and recom- mendation systems (Park, 2019; Gena et al., 2019). ⋄Equal contribution, listed randomly. 1. The genre detection sub-task involves identi- fying the genre of a news article, such as whether it is an opinion piece, a news report, or a satire piece. The task involves a large dataset of texts in different genres and their corresponding genre labels, divided into training, development, and test sets. The evaluation metric is Macro F1-score. Ear- lier research has tackled the challenge of automati- cally detecting satire in English using handcrafted features. As demonstrated by Burfoot and Bald- win (2009), one such feature involves checking the accuracy of entity mentions within the context, and McHardy et al. (2019) proposes an adversarial training method to improve the accuracy of satire detection in news articles. The method addresses the challenge of confounding variables that can influence the classification of a text as satire or non-satire. 2. The news framing sub-task focuses on identi- fying how a news article frames a topic, such as by emphasizing certain …

Content truncated to 500 words