CLaC at SemEval-2023 Task 3: Language Potluck RoBERTa Detects Online Persuasion Techniques in a Multilingual Setup

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1613–1618 July 13-14, 2023 ©2023 Association for Computational Linguistics CLaC at SemEval-2023 Task 3: Language Potluck RoBERTa Detects Online Persuasion Techniques in a Multilingual Setup Nelson Filipe Costa and Bryce Hamilton and Leila Kosseim Computational Linguistics at Concordia (CLaC) Laboratory Department of Computer Science and Software Engineering Concordia University, Montréal, Québec, Canada {nelsonfilipe.costa, bryce.hamilton}@mail.concordia.ca, leila.kosseim@concordia.ca Abstract This paper presents our approach to the SemEval-2023 Task 3 to detect online persua- sion techniques in a multilingual setup. Our classification system is based on the RoBERTa- base model trained predominantly on English to label the persuasion techniques across 9 dif- ferent languages. Our system was able to sig- nificantly surpass the baseline performance in 3 of the 9 languages: English, Georgian and Greek. However, our wrong assumption that a single classification system trained predomi- nantly on English could generalize well to other languages, negatively impacted our scores on the other 6 languages. In this paper, we pro- vide a description of the reasoning behind the development of our final model and what con- clusions may be drawn from its performance for future work. 1 Introduction The SemEval-2023 shared task 3 (Piskorski et al., 2023) focuses on 3 subtasks: detecting the news genre (subtask 1), the framing (subtask 2) and the persuasion techniques (subtask 3) used in online news across different languages. Training data pro- vided by the organizers was collected from news and web articles between the years 2020 and 2022 in six different languages, namely English, French, German, Italian, Polish, and Russian. During the testing phase of the shared task, three additional surprise languages were included only as testing data, Georgian, Greek and Spanish, to explore the language-agnostic robustness of the trained mod- els. In this paper, we focus on the detection of per- suasion techniques used to influence the reader of these online news as part of subtask 3 (Piskorski et al., 2023). We did not participate in the news genre categorisation (subtask 1) and framing detec- tion (subtask 2). In total, 23 different persuasion techniques were annotated at the paragraph-level for each article across all the training languages. Such techniques include loaded language, guilt by association and the straw man fallacy. A more de- tailed description of each persuasion technique is available in the annotation guidelines of the shared task (Piskorski et al., 2023). We applied a RoBERTa (Liu et al., 2019) based model trained predominantly on the English train- ing dataset enriched with specific instances from the other training languages to the detection of persuasion techniques across all languages. Sec- tion 3 provides an overview of our final classifica- tion pipeline. In Section 4, we provide a detailed description of the experiments and data augmenta- tion techniques that led to our final model decision. However, our assumption that a predominately En- glish trained model would be able to generalize well and even outperform models trained on lan- guages specific to the prediction task was proven wrong …

Content truncated to 500 words