Benchmark Dataset for Propaganda Detection in Czech Newspaper Texts

Proceedings of Recent Advances in Natural Language Processing, pages 77–83, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_010 77 Benchmark Dataset for Propaganda Detection in Czech Newspaper Texts V´ıt Baisa and Ondˇrej Herman and Aleˇs Hor´ak Natural Language Processing Centre Masaryk University, Faculty of Informatics Botanick´a 68a, Brno {xbaisa,xherman1,hales}@fi.muni.cz Abstract Propaganda of various pressure groups ranging from big economies to ideological blocks is often presented in a form of ob- jective newspaper texts. However, the real objectivity is here shaded with the sup- port of imbalanced views and distorted at- titudes by means of various manipulative stylistic techniques. In the project of Manipulative Propaganda Techniques in the Age of Internet, a new resource for automatic analysis of stylistic mechanisms for influencing the readers’ opinion is developed. In its current ver- sion, the resource consists of 7,494 news- paper articles from four selected Czech digital news servers annotated for the pres- ence of specific manipulative techniques. In this paper, we present the current state of the annotations and describe the struc- ture of the dataset in detail. We also of- fer an evaluation of bag-of-words classifi- cation algorithms for the annotated manip- ulative techniques. 1 Introduction State and pressure groups propaganda is a very well studied phenomenon from the sociologi- cal point of view (Herman and Chomsky, 2012; Zhang, 2013; Paul and Matthews, 2016). With the spread of digital media, the influence of propa- ganda news grows rapidly (Helmus et al., 2018) and the consequences of public opinion manip- ulation reach new levels (Woolley and Howard, 2017). The main way of self-protection against such propaganda influence lies in careful verification of the presented information sources. Neverthe- less, psycholinguistic evidence (Fazio et al., 2015) shows that a prevailing opinion often outweighs even direct knowledge. Computational tools that could warn against possible manipulation in the text can thus offer an invaluable help even to an informed reader. In the following text, we are presenting the first results of a research project aimed at automatic analysis of the style of a newspaper text to identify a presence of specific manipulative techniques. In the first phase, a specific tool for expert annota- tions of selected news from 4 Czech internet me- dia sites was developed (Baisa et al., 2017). This tool has now been used to obtain 7,494 annotated articles with detailed manipulative techniques an- notations of texts expressing e.g. blaming, demo- nizing, relativizing, labelling, or fear mongering. The following Section 2 provides detailed infor- mation about the dataset characteristics and con- tent. In Section 3, an evaluation of 10 classifica- tion techniques and their results with the bench- mark dataset is presented. 2 The Benchmark Dataset The Propaganda benchmark dataset currently con- tains data from two successive years. The first part is based on two sets of articles from 2016. The newspaper texts were extracted from four newspa- per media domains1 which were previously scru- tinized by annotators as possible sources of pro- Russian propaganda. The downloaded cleaned data were merged with the annotation data stored …

Content truncated to 500 words