NAP at SemEval-2023 Task 3: Is Less Really More? (Back-)Translation as Data Augmentation Strategies for Detecting Persuasion Techniques

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1433–1446 July 13-14, 2023 ©2023 Association for Computational Linguistics NAP at SemEval-2023 Task 3: Is Less Really More? (Back-)Translation as Data Augmentation Strategies for Detecting Persuasion Techniques Neele Falk∗, Annerose Eichel∗, Prisca Piccirilli∗ Institute for Natural Language Processing, University of Stuttgart {firstname.lastname}@ims.uni-stuttgart.de Abstract Persuasion techniques detection in news in a multi-lingual setup is non-trivial and comes with challenges, including little training data. Our system successfully leverages (back- )translation as data augmentation strategies with multi-lingual transformer models for the task of detecting persuasion techniques. The automatic and human evaluation of our aug- mented data allows us to explore whether (back- )translation aid or hinder performance. Our in-depth analyses indicate that both data aug- mentation strategies boost performance; how- ever, balancing human-produced and machine- generated data seems to be crucial. 1 Introduction The SemEval 2023 Task 3 (Piskorski et al., 2023) aims at analyzing online news by detecting genre, framing, and persuasion techniques, i.e., what is presented how using which rhetoric means. Per- suasion techniques detection (Subtask 3) aims to identify which rhetoric means are used to influ- ence and persuade the reader. News articles are provided in English, German, French, Italian, Pol- ish, and Russian. To foster the development of language-agnostic systems like our approach, the organizers additionally introduce three surprise lan- guages – Spanish, Greek, and Georgian1 – with test data only. We build a system that successfully leverages (back-)translation as data augmentation approaches with multi-lingual transformer models to detect persuasion techniques in all 9 languages. We win the task for fr, achieve 2nd place for ge, it and po, and 3rd place for es, el and ka, while ranking mid- field for ru and en. Our main contribution consists in exploring the extent to which data augmenta- tion via (back-)translation boosts performance for *All authors contributed equally to this work. 1Henceforth, we use the following official identifiers: en:English, fr:French, it:Italian, ru:Russian, ge:German, po:Polish, es:Spanish, el:Greek, and ka:Georgian. Figure 1: Example of a multi-labelled paragraph and the corresponding relevant textual spans for the persuasion techniques Loaded Language and Slogans. the task at hand. Our findings suggest that more data does boost performance, especially for under- represented labels. Our in-depth analyses however show that less tends to be really more w.r.t bal- ancing natural vs. augmented data, as more (aug- mented) data can severely hurt performance. 2 Background Predicting a set of persuasion techniques given a piece of news text in a monolingual setting has been explored in previous shared tasks (Da San Martino et al., 2019, 2020; Dimitrov et al., 2021). Existing successful systems usually include monolingual transformer-based models and ensemble mecha- nisms to optimally aggregate predictions (Mapes et al., 2019; Jurkiewicz et al., 2020; Chernyavskiy et al., 2020; Tian et al., 2021). Approaches that additionally focus on provided or external data also show improvement, using techniques such as fine- tuning on additional in-domain data or augmenting the training data (Abujaber et al., 2021). 3 Data Description 3.1 …

Content truncated to 500 words