Fine-Grained Analysis of Propaganda in News Article

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5636–5646, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5636 Fine-Grained Analysis of Propaganda in News Articles Giovanni Da San Martino1 Seunghak Yu2 Alberto Barr´on-Cede˜no3 Rostislav Petrov4 Preslav Nakov1 1 Qatar Computing Research Institute, HBKU, Qatar 2 MIT Computer Science and Artificial Intelligence Laboratory, Cambridge, MA, USA 3 Universit`a di Bologna, Forl`ı, Italy, 4 A Data Pro, Sofia, Bulgaria {gmartino, pnakov}@hbku.edu.qa seunghak@csail.mit.edu, a.barron@unibo.it rostislav.petrov@adata.pro Abstract Propaganda aims at influencing people’s mindset with the purpose of advancing a spe- cific agenda. Previous work has addressed propaganda detection at the document level, typically labelling all articles from a propa- gandistic news outlet as propaganda. Such noisy gold labels inevitably affect the quality of any learning system trained on them. further issue with most existing systems is theA lack of explainability. To overcome these lim- itations, we propose a novel task: performing fine-grained analysis of texts by detecting all fragments that contain propaganda techniques as well as their type. In particular, we cre- ate a corpus of news articles manually anno- tated at the fragment level with eighteen pro- paganda techniques and we propose a suit- able evaluation measure. We further design a novel multi-granularity neural network, and we show that it outperforms several strong BERT-based baselines. 1 Introduction Research on detecting propaganda has focused primarily on articles (Barr´on-Cedeno et al., 2019; Rashkin et al., 2017). In many cases, there are no labeled data for individual articles, but there are such labels for entire news outlets. Thus, often all articles from the same news outlet get labeled the way that this outlet is labeled. Yet, it has been observed that propagandistic sources could post objective non-propagandistic articles periodically to increase their credibility (Horne et al., 2018). Similarly, media generally recognized as objec- tive might occasionally post articles that promote a particular editorial agenda and are thus propagan- distic. Thus, it is clear that transferring the label of the news outlet to each of its articles, could in- troduce noise. Such labels can still be useful for training robust systems, but they cannot be used to get a fair assessment of a system at testing time. One option to deal with the lack of labels for arti- cles is to crowdsource the annotation. However, in preliminary experiments we observed that the av- erage annotator cannot detach her personal mind- set from the judgment of propaganda and bias, i.e., if a clearly propagandistic text expresses ideas aligned with the annotator’s beliefs, it is unlikely that she would judge it as such. We argue that in order to study propaganda in a sound and reliable way, we need to rely on high- quality trusted professional annotations and it is best to do so at the fragment level, targeting spe- cific techniques rather than using a label for an en- tire document or an entire news outlet. Ours is the first …

Content truncated to 500 words