“Don’t discuss”: Investigating Semantic and Argumentative Features for Supervised Propagandist Message Detection and Classification

Proceedings of Recent Advances in Natural Language Processing, pages 1498–1507 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_168 1498 “Don’t discuss”: Investigating Semantic and Argumentative Features for Supervised Propagandist Message Detection and Classification Vorakit Vorakitphan, Elena Cabrio, Serena Villata Universit´e Cˆote d’Azur, Inria, CNRS, I3S, France vorakit.vorakitphan@inria.fr, elena.cabrio@univ-cotedazur.fr, villata@i3s.unice.fr Abstract One of the mechanisms through which dis- information is spreading online, in particular through social media, is by employing pro- paganda techniques. These include specific rhetorical and psychological strategies, rang- ing from leveraging on emotions to exploit- ing logical fallacies. In this paper, our goal is to push forward research on propaganda de- tection based on text analysis, given the cru- cial role these methods may play to address this main societal issue. More precisely, we propose a supervised approach to classify tex- tual snippets both as propaganda messages and according to the precise applied propa- ganda technique, as well as a detailed linguis- tic analysis of the features characterising pro- paganda information in text (e.g., semantic, sentiment and argumentation features). Ex- tensive experiments conducted on two avail- able propagandist resources (i.e., NLP4IF’19 and SemEval’20-Task 11 datasets) show that the proposed approach, leveraging different language models and the investigated linguis- tic features, achieves very promising results on propaganda classification, both at sentence- and at fragment-level. 1 Introduction Propaganda represents an effective, even though of- ten misleading, communication strategy to promote a cause or a viewpoint, for instance in the political context (Lasswell, 1938; Koppang, 2009; Dillard and Pfau, 2009; Longpre et al., 2019). Different communication means can be used to disseminate propaganda, i.e., textual documents, images, videos and oral speeches. The ability to effectively iden- tify and manifestly label such kind of misleading and potentially harmful content is of primarily im- portance to restrain the spread of such information to avoid detrimental consequences for the society. In this paper, we tackle this challenging is- sue (Da San Martino et al., 2020b) by proposing a textual propaganda detection model. More pre- cisely, we address the following research questions: (i) how to automatically identify propaganda in tex- tual documents and further classify them into fine- grained categories?, and (ii) what are the linguistic distinctive features of propaganda text snippets? The contribution of this paper consists not only in proposing a new effective neural architecture to au- tomatically identify and classify propaganda in text, but we also present a detailed linguistic analysis of the features characterising propaganda messages. Our work focuses on the propaganda detection and classification task, casting it both as a binary and as a multi-class classification task, and we ad- dress it both at sentence-level and at fragment-level. We investigate different architectures of recent lan- guage models (i.e., BERT, RoBERTa), combining them with a rich set of linguistic features ranging from sentiment and emotion to argumentation fea- tures, to rhetorical stylistic ones. The extensive experiments we conducted on two standard bench- marks (i.e., the NLP4IF’19 and SemEval’20-Task 11 datasets) show that the proposed architectures achieve satisfying results, outperforming state-of- the-art systems on most of …

Content truncated to 500 words