Team DoNotDistribute at SemEval-2020 Task 11: Features, Finetuning, and Data Augmentation in Neural Models for Propaganda Detection in News Articles

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1502–1508 Barcelona, Spain (Online), December 12, 2020. 1502 Team DoNotDistribute at SemEval-2020 Task 11: Features, Finetuning, and Data Augmentation in Neural Models for Propaganda Detection in News Articles Michael Kranzlein, Shabnam Behzad, Nazli Goharian Department of Computer Science Georgetown University {mmk119, sb1796}@georgetown.edu nazli@ir.cs.georgetown.edu Abstract This paper presents our systems for SemEval 2020 Shared Task 11: Detection of Propaganda Techniques in News Articles. We participate in both the span identification and technique classification subtasks and report on experiments using different BERT-based models along with handcrafted features. Our models perform well above the baselines for both tasks, and we contribute ablation studies and discussion of our results to dissect the effectiveness of different features and techniques with the goal of aiding future studies in propaganda detection. 1 Introduction Propagandist news articles are misleading in nature and aim at biasing their audience towards a particular point of view by using psychological and rhetorical techniques, including loaded language, name calling, repetition, exaggeration, minimization, etc. With the rapid growth in the number of online sources of information and the speed with which information spreads online, manual flagging of propagandist news articles has become untenable, leading to an ongoing need for new research on methods for identifying these articles automatically to mitigate the negative influence they might have on users. Until very recently, most of the work in this area has focused on article-level detection (Rashkin et al., 2017; Barrón-Cedeño et al., 2019a). However, in 2019, Da San Martino et al. (2019b) published a corpus of English news articles with individual spans of propaganda annotated that addresses the problem at a more granular level. This corpus is used in shared tasks at NLP4IF-2019 (Da San Martino et al., 2019a) and at SemEval-2020 (Da San Martino et al., 2020). The 2020 shared task is a modified version of the prior year’s task and includes two subtasks1: 1. Span Identification (SI): Given a plain-text document, identify those specific fragments which contain at least one propaganda technique. This is a binary sequence tagging task. 2. Technique Classification (TC): Given a text fragment identified as propaganda and its document context, identify the applied propaganda technique in the fragment. We present our models2 for both tasks alongside discussions of our results and ablations. 2 Background Automatic fact-checking and propaganda detection in news articles have attracted considerable attention in recent years (Potthast et al., 2018; Helmstetter and Paulheim, 2018; Baly et al., 2018). Many of these works focus on identifying propaganda at the article level. Rashkin et al. (2017) created a corpus of news articles with 4 different news types: Trusted, Satire, Hoax and Propaganda. They compared the language of these different types and reported classification results using LSTM, Max-Ent, and Naive Bayes models. Barrón-Cedeño et al. (2019b) experimented on this corpus with a maximum entropy classifier to discriminate propagandist from non-propagandist articles. 1https://propaganda.qcri.org/semeval2020-task11/ 2Model hyperparameters, code, and instructions for reproducing our results are available at https://github.com/ mkranzlein/propaganda This work is licensed …

Content truncated to 500 words