PsuedoProp at SemEval-2020 Task 11: Propaganda Span Detection Using BERT-CRF and Ensemble Sentence Level Classifier

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1779–1785 Barcelona, Spain (Online), December 12, 2020. 1779 PsuedoProp at SemEval-2020 Task 11:Propaganda Span Detection using BERT-CRF and Ensemble Sentence Level Classifier Aniruddha Chauhan, Harshita Diddee Department of Computer Science and Engineering Bharati Vidyapeeth’s College of Engineering New Delhi, India aniruddhac7@gmail.com, harshita.bvcoend@bvp.edu.in Abstract This paper presents our solution for Span Identification (SI) Task under “Task 11: Detection of Propaganda Techniques in News Articles” of SemEval 2020. This task aims to identify if a given sentence, taken from a corpus of news articles, contains a propaganda span and hence aims to identify the character level offsets of the identified propaganda element. Our solution proposes a sequential approach in which the span identification is preceded by an ensemble sentence level classifier (SLC). We only perform span identification on those samples which are flagged as propaganda samples by the SLC Model. We perform token level classification by fine-tuning BERT and use CRF to perform sequence tagging. Additionally, we present our analysis of different voting ensembles for the SLC model. Our system ranks 14th on the test set and 22nd on the development set and with an F1 score of 0.41 and 0.39 respectively. 1 Introduction In contemporary times, fake news and propaganda have gained a lot of traction. A contributing factor to these problems is the easy dissemination of information on social media and various alternative news outlets on the Internet which house a vast repository of content which is tough to effectively moderate. Propaganda is often used to promulgate news articles or content that is misleading. In conjunction, Fake News not only contrives hysteria and spreads lies, in extreme cases it leads to physical violence (Kang and Goldman, 2016). Most of the workaround propaganda detection has been limited to document-level classification (Shu et al., 2017; Barr´on-Cedeno et al., 2019; Rashkin et al., 2017). In the past, Shared tasks such as the NLP4IF 2019 have dealt with Sentence Level Classification (SLC) and Fragment Level Classification (FLC) of propaganda (Da San Martino et al., 2019). Fine-grained propaganda techniques provide a more suitable method of detecting propaganda because its classification provides the reasoning behind why an instance has been flagged as propaganda. The SemEval shared Task 11 makes progress in this aspect with its two tasks namely, Span Identification (SI) that has the objective of finding propaganda spans, and Technique Classification (TC) that labels the propaganda technique employed in a propaganda span (Da San Martino et al., 2020). In this paper, we have focused on the Span Identification task that involves character level tagging of propaganda spans in text. To achieve this, we used ensemble transformer-based architectures to first perform SLC which is followed by token level tagging of spans of only propaganda sentences. These are fed to the BERT-CRF span identification model, predictions of which are later processed to obtain the character level tagging of propaganda fragments. In addition to this, we carry out various experiments to deal with the class imbalance in the …

Content truncated to 500 words