SheffieldVeraAI at SemEval-2023 Task 3: Mono and Multilingual Approaches for News Genre, Topic and Persuasion Technique Classification

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1995–2008 July 13-14, 2023 ©2023 Association for Computational Linguistics SheffieldVeraAI at SemEval-2023 Task 3: Mono and multilingual approaches for news genre, topic and persuasion technique classification Ben Wu#, Olesya Razuvayevskaya#, Freddy Heppell#, João A. Leite#, Carolina Scarton, Kalina Bontcheva, and Xingyi Song Department of Computer Science, The University of Sheffield, Sheffield, UK {bpwu1, o.razuvayevskaya, frheppell1, jaleite1}@sheffield.ac.uk Abstract This paper describes our approach for SemEval- 2023 Task 3: Detecting the category, the fram- ing, and the persuasion techniques in online news in a multilingual setup. For Subtask 1 (News Genre), we propose an ensemble of fully trained and adapter mBERT models which was ranked joint-first for German, and had the high- est mean rank of multi-language teams. For Subtask 2 (Framing), we achieved first place in 3 languages, and the best average rank across all the languages, by using two separate ensem- bles: a monolingual RoBERTa-MUPPETLARGE and an ensemble of XLM-RoBERTaLARGE with adapters and task adaptive pretraining. For Sub- task 3 (Persuasion Techniques), we trained a monolingual RoBERTa-Base model for English and a multilingual mBERT model for the re- maining languages, which achieved top 10 for all languages, including 2nd for English. For each subtask, we compared monolingual and multilingual approaches, and considered class imbalance techniques. 1 1 Introduction With the rise of opinion-manipulating news and misinformation surrounding COVID-19, elections and wars, the task of propaganda and hyperpartisan detection has received much attention over the last five years. Since 2019, various SemEval tasks have addressed detecting hyperpartisan (Kiesel et al., 2019), sarcasm (Abu Farha et al., 2022), and per- suasion techniques in textual and multimodal data (Da San Martino et al., 2020; Dimitrov et al., 2021). This task (Piskorski et al., 2023) can be seen as an extension of the latter two tasks, suggesting an expanded ontology of persuasion techniques and addressing other related aspects of persuasion, such as satire, opinionated news, and framing detection. #Equal contribution, listed randomly. 1Our code is available at https://github.com/ GateNLP/semeval2023-multilingual-news-detection The three subtasks presented in this shared task are the detection of: 1) genre: opinion, objective reporting or satire; 2) framing techniques: 14 mul- tilabel frames; 3) persuasion techniques: 23 mul- tilabel techniques, which can be grouped into 6 high-level classes. The data consists of labelled training and de- velopment sets in English, French, German, Ital- ian, Polish and Russian, and unlabelled test sets in the same languages plus three zero-shot languages: Spanish, Greek and Georgian. The main contributions of this paper are twofold: 1) evaluation of the viability of monolingual versus multilingual models for each of the subtasks; and 2) presentation of the models which ranked first in four subtask-language pairs, top three in 16 subtask- language pairs, and were within the top 10 for all. Our approaches for the three subtasks differ, therefore we present each subtask separately in sections 3-5 respectively. An overview of the tech- niques used in each subtask is shown in Table 1. 2 Background Fine-grained propaganda …

Content truncated to 500 words