Inno at SemEval-2020 Task 11: Leveraging Pure Transfomer for Multi-Class Propaganda Detection

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1481–1487 Barcelona, Spain (Online), December 12, 2020. 1481 Inno at SemEval-2020 Task 11: Leveraging Pure Transfomer for Multi-Class Propaganda Detection Dmitry Grigorev Innopolis University d.grigorev@innopolis.ru Vladimir Ivanov Innopolis University v.ivanov@innopolis.ru Abstract The paper presents the solution of team ”Inno” to a SEMEVAL 2020 task 11 ”Detection of propaganda techniques in news articles”. The goal of the second subtask is to classify textual segments that correspond to one of the 18 given propaganda techniques in news articles dataset. We tested a pure Transformer-based model with an optimized learning scheme on the ability to distinguish propaganda techniques between each other. Our model showed 0.6 and 0.58 overall F1 score on validation set and test set accordingly and non-zero F1 score on each class on both sets. 1 Introduction Modern society is experiencing an overflow of different kinds of information. Thousands of news media are publishing a giant number of articles, blogposts, twits and other kind of content every day and even every hour. People who want to stay informed about global situation are often not able to distinguish which source is reliable and provides relevant content. Therefore, such topics as automatic fake news detection, sentiment analysis and propaganda detection are gaining increasing attention. The authors of (Jowett and O’donnell, 2018) defined propaganda in the most neutral sense as dissem- ination or promotion of a particular idea. This paper provides our solution to SEMEVAL 2020 task 11 “Detection of propaganda techniques in news articles”. The task consists of two subtasks: (i) a propaganda sample span identification and bound- ing and (ii) a classification of these spans into one of the 18 classes (or techniques). Some techniques in the second subtask were merged into one class to get 14 classes for prediction in the end. All the tech- niques, as well as the other details about the task, are described by the task organizers in (Da San Martino et al., 2020). In this paper we present a solution to the second subtask. The suggested system1 consists of a Transformer-based classifier with additional learning optimization techniques such as undersampling, cost-sensitive learning and context addition, aimed at robustness. This approach allowed us to achieve 0.6 and 0.58 overall F1 scores with non-zero F1 score for each class on validation set and test set accord- ingly. We took the 7th place out of 31 teams on the second subtask. 2 Background One of the first classifications of propaganda techniques was proposed in 1936 by Clyde R. Miller (Ed- wards, 1938) after a presidential election in the USA. This categorization consists of description of 7 propaganda techniques: “Name calling”, “Glittering generalities”, “Transfer”, “Testimonial”, “Plain folks”, “Card stacking” and “Band wagon”. Since then a lot more techniques have been identified, but this classification is still relevant (some of the techniques have been included in the SEMEVAL 2020). Probably, the most promising state-of-art strategies of dealing with propaganda and filtering it from senti- mentally neutral texts is the one of …

Content truncated to 500 words