UPB at SemEval-2020 Task 11: Propaganda Detection with Domain-Specific Trained BERT

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1853–1857 Barcelona, Spain (Online), December 12, 2020. 1853 UPB at SemEval-2020 Task 11: Propaganda Detection with Domain-Specific Trained BERT Andrei Paraschiv, Dumitru-Clementin Cercel, Mihai Dascalu University Politehnica of Bucharest, Faculty of Automatic Control and Computers andrei.paraschiv74@stud.acs.pub.ro {dumitru.cercel, mihai.dascalu}@upb.ro Abstract Manipulative and misleading news have become a commodity for some online news outlets and these news have gained a significant impact on the global mindset of people. Propaganda is a frequently employed manipulation method having as goal to influence readers by spreading ideas meant to distort or manipulate their opinions. This paper describes our participation in the SemEval-2020, Task 11: Detection of Propaganda Techniques in News Articles competition. Our approach considers specializing a pre-trained BERT model on propagandistic and hyperpartisan news articles, enabling it to create more adequate representations for the two subtasks, namely propaganda Span Identification (SI) and propaganda Technique Classification (TC). Our proposed system achieved a F1-score of 46.060% in subtask SI, ranking 5th in the leaderboard from 36 teams and a micro-averaged F1 score of 54.302% for subtask TC, ranking 19th from 32 teams. 1 Introduction Propagandistic texts can be perceived as deliberate acts of spreading ideas with the self-serving purpose of influencing, distorting, and manipulating the opinions of other individuals or groups (Da San Martino et al., 2019). Propagandistic texts use several techniques to achieve these goals - for example, using emotionally charged statements to persuade the audience not on the merits of the argument, but based on the visceral reaction to it. Thus, early detection and the labeling of such texts is of utmost importance to lessen the negative impact on the society and diminish the spread of such news through online social media. SemEval-2020 Task 11 proposes the detection of propagandistic text sequences in English news articles, including a span detection subtask (subtask SI), followed by a fine-grained propaganda classification into 14 classes (subtask TC) (Da San Martino et al., 2020). The competition was divided into three phases. Participants had access to the training set with full golden labels and a dev set without the golden labels. During the development phase, participants could submit the results on the dev set for both the SI and TC subtasks, and receive an instant score. In contrast, the test set submissions received the scores only at the end of the competition. Our solution builds upon BERT (Devlin et al., 2019), a pre-trained Transformer-based (Vaswani et al., 2017) language model composed of several (12 or 24) multi-head attention blocks. The pre-trained model provided by Google was further pre-trained to create a domain-specific model using masked language modeling on a corpus consisting of news articles. This approach was used in narrow domains, like scientific papers (Beltagy et al., 2019) or bio-medical data (Lee et al., 2020), with significant performance gain in domain-specific tasks. The rest of the paper is organized as follows. Section 2 presents work related to the propaganda detection problem. Section 3 provides details on the dataset, its …

Content truncated to 500 words