HOMADOS at SemEval-2021 Task 6: Multi-Task Learning for Propaganda Detection

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1027–1031 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1027 HOMADOS at SemEval-2021 Task 6: Multi-Task Learning for Propaganda Detection Konrad Kaczy´nski1,2 and Piotr Przybyła1 1Institute of Computer Science, Polish Academy of Sciences 2Faculty of Economic Sciences, University of Warsaw {konrad.kaczynski,piotr.przybyla}@ipipan.waw.pl Abstract Among the tasks motivated by the prolifera- tion of misinformation, propaganda detection is particularly challenging due to the deficit of fine-grained manual annotations required to train machine learning models. Here we show how data from other related tasks, includ- ing credibility assessment, can be leveraged in multi-task learning (MTL) framework to accel- erate the training process. To that end, we de- sign a BERT-based model with multiple output layers, train it in several MTL scenarios and perform evaluation against the SemEval gold standard. 1 Introduction Fine-grained propaganda detection is a new ap- proach to tackling online misinformation, highlight- ing instances of propaganda techniques on the word level. These techniques are used in textual com- munication in order to encourage certain beliefs, but instead of straightforward presentation of ar- guments, they rely on psychological manipulation, logical fallacies or emotion elicitation. There are general-purpose natural language pro- cessing (NLP) methods that could be used for auto- matic detection of such text fragments. The chal- lenge here is that they require large amounts of training data, which are laborious to produce. How- ever, propaganda techniques are often related to other misinformation challenges, for which large datasets do exist, e.g. credibility assessment or fake news detection. In the present study we aim to investigate how this connection can be used in the multi-task learn- ing (MTL) framework. We show how the per- formance of multi-label token-level propaganda detection within shared task 6 at SemEval-2021 can be improved by building neural architectures that are also trained to solve other tasks: single- label propaganda detection from SemEval-2020 and document-level credibility assessment based on a fake news corpus. We check different MTL scenarios (parallel and sequential) and show which aspects of the model benefit the most from this approach. 2 Problem Statement We participate in SemEval-2021 Task 6 (,,Detec- tion of Persuasion Techniques in Texts and Im- ages”), subtask 2 (Dimitrov et al., 2021), where the goal is to identify all propaganda techniques within a provided fragment of text. Specifi- cally, given a character sequence ⟨c0, c1, . . . , cN⟩, we aim to produce a set of annotations {(b0, e0, t0), (b1, e1, t1), . . . , (bk, ek, tk)}, where each triple consists of the character offsets of the span it covers (0 ≤ bi < ei ≤ N) and an indi- cation which one from the set of 20 known tech- niques is detected there (ti ∈ T). We can see it as a multi-label sequence classification task (Read et al., 2009), where each character (or token) can be assigned from 0 to 20 labels. 3 Related Work Propaganda has been observed in text for a …

Content truncated to 500 words