FPAI at SemEval-2021 Task 6: BERT-MRC for Propaganda Techniques Detection

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1056–1060 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1056 FPAI at SemEval-2021 Task 6: BERT-MRC for Propaganda Techniques Detection Xiaolong Hou∗, Junsong Ren∗, Gang Rao, Lianxin Jiang, Zhihao Ruan, Yang Mo, Jianping Shen Ping An Life Insurance, Lt ShenZhen, China {houxiaolong430, renjunsong941, raogang532, jianglianxin769, ruanzhihao332, moyang853, shenjianping324}@pingan.com.cn Abstract The objective of subtask 2 of SemEval-2021 Task 6 is to identify techniques used together with the span(s) of text covered by each tech- nique. This paper describes the system and model we developed for the task. We first propose a pipeline system to identify spans, then to classify the technique in the input se- quence. But it severely suffers from handling the overlapping in nested span. Then we pro- pose to formulize the task as a question an- swering task by machine reading comprehen- sion (MRC) framework which achieves a bet- ter result compared to the pipeline method. Moreover, data augmentation and loss design techniques are also explored to alleviate the problem of data sparsity and imbalance. Fi- nally, we attain the 3rd place in the final evalu- ation phase. 1 Introduction Fake news detection has attracted the attention of many researchers. But most of the conventional fake news detection methods focuse on long-form journalism, and little attention has been paid to the propaganda techniques. Memes have become the most popular type of content on social media plat- forms, and it can easily mislead the audience to agree with the speaker through propaganda tech- niques. The SemEval-2021 Task 6 focuses on de- tecting the use of rhetorical and psychological tech- niques in memes without (subtask 1, subtask 2) and with (subtask 3) visual content. We first adopted a model to identify the span and techniques sequentially, and made many attempts to optimize the model, but the results were not sat- isfactory. Then we tried some end-to-end method, e.g. MRC framework. ∗These authors contributed equally to this work This work is licensed under a Creative Commons Attribution 4.0 In- ternational License. License details: http:// creativecom- mons.org/licenses/by/4.0/ We found that the pipeline model can’t handle the span overlapping issue effectively, but the MRC framework with an informative query is well per- forming in this scenario. In addition, the data aug- mentation and a carefully designed loss can allevi- ate the impact of data sparsity and imbalance. We attain an F1 score of 0.3974 and the 3rd place in the final evaluation phase 2 Related Work There was also a related previous task on fine- grained propaganda detection (Da San Mar- tino et al., 2019), where the participants used Transformer-style models, LSTMs and ensembles (Fadel et al.,2019;Hou and Chen,2019;Hua,2019). Some approaches further used non-contextualized word embeddings, e.g., based on FastText and GloVe (Gupta et al.,2019; Al-Omari et al., 2019), or handcrafted features such as LIWC, quotes and questions (Alhindi et al., 2019). Moreover, Mar- tino et al.2020 analysed computational propaganda detection from Text Perspective and Network Per- spective, argued …

Content truncated to 500 words