LTIatCMU at SemEval-2020 Task 11: Incorporating Multi-Level Features for Multi-Granular Propaganda Span Identification

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1756–1763 Barcelona, Spain (Online), December 12, 2020. 1756 LTIatCMU at SemEval-2020 Task 11: Incorporating Multi-Level Features for Multi-Granular Propaganda Span Identification Sopan Khosla∗ Rishabh Joshi∗ Ritam Dutt∗ Alan W Black Yulia Tsvetkov Language Technology Institute Carnegie Mellon University {sopank,rjoshi2,rdutt,awb,ytsvetko}@cs.cmu.edu Abstract In this paper we describe our submission for the task of Propaganda Span Identification in news articles. We introduce a BERT-BiLSTM based span-level propaganda classification model that identifies which token spans within the sentence are indicative of propaganda. The “multi- granular” model incorporates linguistic knowledge at various levels of text granularity, includ- ing word, sentence and document level syntactic, semantic and pragmatic affect features, which significantly improve model performance, compared to its language-agnostic variant. To facili- tate better representation learning, we also collect a corpus of 10k news articles, and use it for fine-tuning the model. The final model is a majority-voting ensemble which learns different propaganda class boundaries by leveraging different subsets of incorporated knowledge.1 1 Introduction Propaganda (Bernays, 1928) is “the deliberate and systematic attempt to shape perceptions, manipulate cognition, and direct behavior to achieve a response that furthers the desired intent of the propagandist” (Jowett and O’donnell, 2018). The rise of digital and social media has enabled propaganda campaigns to reach vast audiences (Glowacki et al., 2018; Tardáguila et al., 2018), influencing or undermining the democratic processes, amplifying the effects of misinformation, and creating echo-chambers. As a tactic of manipulation, propaganda is considered most effective when it is veiled: the read- ers are not able to identify it, but their opinions are shaped according to the propagandist’s hidden agenda (Barrón-Cedeño et al., 2019); it is thus very difficult to identify propaganda automatically. How- ever, automatic identification and analysis of propaganda in news articles and on social media are es- sential to understand propaganda at scale and develop approaches to countering it (King et al., 2017; Starbird, 2018; Field et al., 2018). Prior research on propaganda detection focused primarily on identifying propaganda at a document level, due to the dearth of finer-grained labelled data (Rashkin et al., 2017; Barrón-Cedeño et al., 2019). This has resulted in classifying all news articles from a particular source as propaganda (or hoax, disin- formation, etc.), which is often not the case, and which can obfuscate a finer-grained analysis of propa- gandistic strategies (Da San Martino et al., 2019; Horne et al., 2018). Recently, Da San Martino et al. (2019) carried out a seminal task of fine-grained propaganda detec- tion and curated a dataset consisting of about 550 news articles. The dataset contains word-span level annotations provided by high-quality professionals along with additional information about the propa- ganda technique employed in the span. The Span Identification (SI) sub-task of SemEval 2020, Task 11, (Da San Martino et al., 2020) employs this aforementioned corpus and requires participants to detect propaganda spans in news articles. In this paper, we describe our solution to the SI task. We propose a BERT-BiLSTM based multi- granularity model, inspired …

Content truncated to 500 words