DSHacker at SemEval-2023 Task 3: Genres and Persuasion Techniques Detection with Multilingual Data Augmentation through Machine Translation and Text Generation

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1582–1591 July 13-14, 2023 ©2023 Association for Computational Linguistics DSHacker at SemEval-2023 Task 3: Genres and Persuasion Techniques Detection with Multilingual Data Augmentation through Machine Translation and Text Generation Arkadiusz Modzelewski ∗ Witold Sosnowski ∗ Magdalena Wilczynska Adam Wierzbicki Polish-Japanese Academy of Information Technology Koszykowa 86, 02-008 Warsaw, Poland firstname.lastname@pja.edu.pl Abstract In our article, we present the systems developed for SemEval-2023 Task 3, which aimed to eval- uate the ability of Natural Language Processing (NLP) systems to detect genres and persuasion techniques in multiple languages. We exper- imented with several data augmentation tech- niques, including machine translation (MT) and text generation. For genre detection, synthetic texts for each class were created using the Ope- nAI GPT-3 Davinci language model. In con- trast, to detect persuasion techniques, we relied on augmenting the dataset through text trans- lation using the DeepL translator. Fine-tuning the models using augmented data resulted in a top-ten ranking across all languages, indicating the effectiveness of the approach. The mod- els for genre detection demonstrated excellent performance, securing the first, second, and third positions in Spanish, German, and Ital- ian, respectively. Moreover, one of the models for persuasion techniques’ detection secured the third position in Polish. Our contribution constitutes the system architecture that utilizes DeepL and GPT-3 for data augmentation for the purpose of detecting both genre and persuasion techniques. 1 Introduction The ability to detect genres 1 and persuasion tech- niques in the text is crucial in NLP, as it allows to better evaluate the quality of information and categorize it. The SemEval-2023 2 Task 3 is a shared task that aims to evaluate the performance of systems in detecting genres, framing, and persua- sion techniques in multiple languages. This paper presents our systems for detecting genre and per- ∗Equal contribution. 1By genre we mean satire, opinion and reporting 2The 17th International Workshop on Semantic Evaluation. https://semeval.github.io/SemEval2023/ suasion techniques3, which was prepared for that workshop. Our methods involve machine translation and text generation techniques to augment the original training data in all participating languages. Specifi- cally, we augmented the genre detection datasets in these languages by creating synthetic texts for each class with the OpenAI GPT-3 Davinci language model. Afterward, we fine-tuned a single multilin- gual XLM-RoBERTa-large model for genre detec- tion for all languages. Our approach to building systems to detect different persuasion techniques involved a combination of feature engineering and deep learning models. We primarily utilized the DeepL 4 translator for text translation to augment the dataset. Our final systems performed well in the SemEval-2023 Task 3, ranking among the top 10 participants across all languages. We structured our paper as follows: In Section 2, we give a summary of prior work in the field of de- tecting genres and persuasion techniques. Sections 3 and 4 describe the methods we used for data aug- mentation and our systems architecture. Sections 5 and 6 present our experimental setup and the results we obtained. Section …

Content truncated to 500 words