SocCogCom at SemEval-2020 Task 11: Characterizing and Detecting Propaganda Using Sentence-Level Emotional Salience Features

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1793–1801 Barcelona, Spain (Online), December 12, 2020. 1793 SocCogCom at SemEval-2020 Task 11: Characterizing and Detecting Propaganda using Sentence-Level Emotional Salience Features Gangeshwar Krishnamurthy, Raj Kumar Gupta, Yinping Yang Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore {gangeshwark,gupta-rk,yangyp}@ihpc.a-star.edu.sg Abstract This paper describes a system developed for detecting propaganda techniques from news articles. We focus on examining how emotional salience features extracted from a news segment can help to characterize and predict the presence of propaganda techniques. Correlation analyses surfaced interesting patterns that, for instance, the “loaded language” and “slogan” techniques are negatively associated with valence and joy intensity but are positively associated with anger, fear and sadness intensity. In contrast, “flag waving” and “appeal to fear-prejudice” have the exact opposite pattern. Through predictive experiments, results further indicate that whereas BERT-only features obtained F1-score of 0.548, emotion intensity features and BERT hybrid features were able to obtain F1-score of 0.570, when a simple feedforward network was used as the classifier in both settings. On gold test data, our system obtained micro-averaged F1-score of 0.558 on overall detection efficacy over fourteen propaganda techniques. It performed relatively well in detecting “loaded language” (F1 = 0.772), “name calling and labeling” (F1 = 0.673), “doubt” (F1 = 0.604) and “flag waving” (F1 = 0.543). 1 Introduction Propaganda is studied in a wide range of social sciences disciplines, including social psychology, political science, media and mass communication, as well as advertising and marketing (Davison, 1971; Taylor, 2002; Balfour, 1979; McGarry, 1958). As Jowett and O’Donnell (2018) put it, propaganda is a “deliberate, systematic attempt to shape perceptions, manipulate cognitions, and direct behavior to achieve a response that furthers the desired intent of the propagandist”. To achieve the agenda, propagandists may use various influence techniques such as loaded emotive language and flag waving. Such techniques are centered on influencing the audiences’ opinions and behaviors through psychological and rhetorical tricks in order to reach its purpose, such as promoting a particular politician or product in political or marketing campaigns. The ability to automatically detect propaganda has important societal implications. For news manage- ment, propaganda detection may help publishers to quickly identify news articles that may be subjected to propagandistic characteristics that severely deviate from journalism principles. For the general public, such tools may raise awareness for social media users to stay alert of potential propagandistic content, which often may leverage non-obvious psychological tricks, and potentially mitigate the propagation of such content. We participated in Task 11 on the detection of propaganda techniques in news articles (Da San Martino et al., 2020a), in particular the Technique Classification task (task TC), a multi-class classification task that aims to classify each identified text segment with the existence of a collection of fourteen propaganda techniques (Da San Martino et al., 2020a). Appendix A provides a summary and a distribution analysis on this task. This text segment-based ground truth data presents an advancement to this line of …

Content truncated to 500 words