Team DiSaster at SemEval-2020 Task 11: Combining BERT and Hand-crafted Features for Identifying Propaganda Techniques in News

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1817–1822 Barcelona, Spain (Online), December 12, 2020. 1817 Team DiSaster at SemEval-2020 Task 11: Combining BERT and hand-crafted Features for Identifying Propaganda Techniques in News Media Anders Friis Kaas IT University of Copenhagen Rued Langgaards Vej 7 2300 Copenhagen anfk@itu.dk Viktor Torp Thomsen IT University of Copenhagen Rued Langgaards Vej 7 2300 Copenhagen vikt@itu.dk Barbara Plank IT University of Copenhagen Rued Langgaards Vej 7 2300 Copenhagen bapl@itu.dk Abstract The identification of communication techniques in news articles such as propaganda is important, as such techniques can influence the opinions of large numbers of people. Most work so far focused on the identification at the news article level. Recently, a new dataset and shared task has been proposed for the identification of propaganda techniques at the finer-grained span level. This paper describes our system submission to the subtask of technique classification (TC) for the SemEval 2020 shared task on detection of propaganda techniques in news articles. We propose a method of combining neural BERT representations with hand-crafted features via stacked generalization. Our model has the added advantage that it combines the power of contextual representations from BERT with simple span-based and article-based global features. We present an ablation study which shows that even though BERT representations are very powerful also for this task, BERT still benefits from being combined with carefully designed task-specific features. 1 Introduction The purpose of propaganda is to use communication to foster predetermined agendas, or to achieve a response that furthers a desired outcome (Jowett and O’Donnell, 2018). Prior research has focused on creating machine learning models that label whole news articles or even entire news outlets as propagandistic (Rashkin et al., 2017; Barr´on-Cede˜no et al., 2019). To increase the granularity of these coarse models, a new data set was developed in a study by Da San Martino et al (2019), which enabled models to jointly identify fragments of propaganda within a document, while also classifying their respective propaganda techniques (Da San Martino et al., 2019; Yu et al., 2019). This paper presents our solution (DiSaster, finishing at 11th place) to the technique classification (TC) sub-task at SemEval 2020 task 11 “Detection of propaganda techniques in news articles”.1 TC is a multi class classification problem in which a system needs to identify the propaganda techniques of a given span of an article. For instance, when given the span “stupid and petty” the system should classify it as Loaded Language. Our system is an ensemble model based on stacked generalization (Wolpert, 1992) which enables the incorporation of both traditional engineered features (Nalini and Sheela, 2014) and the Transformer (Vaswani et al., 2017) based language model BERT (Devlin et al., 2019). 2 Related work In addition to formulating the original problem of fine-grained propaganda identification and creating the corpus needed to solve the task, Da San Martino et al. (2019) also designed a multi-granularity neural network. This model outperformed several strong BERT baseline models in the high granularity fragment-level classification …

Content truncated to 500 words