NTUAAILS at SemEval-2020 Task 11: Propaganda Detection and Classification with biLSTMs and ELMo

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1495–1501 Barcelona, Spain (Online), December 12, 2020. 1495 NTUAAILS at SemEval-2020 Task 11: Propaganda Detection and Classification with biLSTMs and ELMo Anastasios Arsenos, Georgios Siolas Artificial Intelligence and Learning Systems Laboratory, National Technical University of Athens Zografou, Athens, Greece anarsenos@gmail.com, gsiolas@islab.ntua.gr Abstract This paper describes the NTUAAILS submission for SemEval 2020 Task 11 Detection of Pro- paganda Techniques in News Articles. This task comprises of two different sub-tasks, namely A: Span Identification (SI), B: Technique Classification (TC). The goal for the SI sub-task is to identify specific fragments, in a given plain text, containing at least one propaganda technique. The TC sub-task aims to identify the applied propaganda technique in a given text fragment. A different model was trained for each sub-task. Our best performing system for the SI task consists of pre-trained ELMo word embeddings followed by residual bidirectional LSTM network. For the TC sub-task pre-trained word embeddings from GloVe fed to a bidirectional LSTM neural network. The models achieved rank 28 among 36 teams with F1 score of 0.335 and rank 25 among 31 teams with 0.463 F1 score for SI and TC sub-tasks respectively. Our results indicate that the proposed deep learning models, although relatively simple in architecture and fast to train, achieve satisfactory results in the tasks on hand. 1 Introduction Propaganda is the expression of an opinion or an action by individuals or groups deliberately designed to influence the opinions or the actions of other individuals or groups with reference to predetermined ends (Miller, C. R, 1937 1938). It is often combined with misinformation and fake news and all together have the potential to polarise public opinion, to promote violent extremism and hate speech. To deal with this problem, automatic identification and categorisation of propaganda, fake news and hyperpartisan content has been heavily addressed in recent research, mainly in the article level (Da San Martino et al., 2019a; Rashkin et al., 2017; Kiesel et al., 2019). This task is a follow-up to the NLP4IF shared task on fine-grained propaganda detection(Da San Martino et al., 2019c) and aims to produce models capable of spotting propaganda techniques in text fragments and then categorise them in one or more of 14 propaganda techniques. The first sub-task is called Span identification (SI) and its goal is to spot propaganda fragments given a plain text document. The evaluation metric for this task is a modified F1 measure appropriate for taking into account partial matching between the spans. Detailed information about this metric can be found in (Da San Martino et al., 2019b; Da San Martino et al., 2020). The second sub-task which is called Technique classification (TC) consists of text fragments as inputs, labelled as one or more of 14 propaganda classes such as Name Calling and Loaded Language. The goal is to classify the fragments to one or more of those techniques. Due to the fact that the distribution of the techniques in gold labels is rather imbalanced, results …

Content truncated to 500 words