Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1075–1081 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1075 Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble Kshitij Gupta∗ Devansh Gautam∗ Radhika Mamidi International Institute of Information Technology Hyderabad {kshitij.gupta,devansh.gautam}@research.iiit.ac.in, radhika.mamidi@iiit.ac.in Abstract Memes are one of the most popular types of content used to spread information online. They can influence a large number of peo- ple through rhetorical and psychological tech- niques. The task, Detection of Persuasion Techniques in Texts and Images, is to detect these persuasive techniques in memes. It con- sists of three subtasks: (A) Multi-label classi- fication using textual content, (B) Multi-label classification and span identification using tex- tual content, and (C) Multi-label classification using visual and textual content. In this pa- per, we propose a transfer learning approach to fine-tune BERT-based models in different modalities. We also explore the effectiveness of ensembles of models trained in different modalities. We achieve an F1-score of 57.0, 48.2, and 52.1 in the corresponding subtasks. 1 Introduction Memes are text superimposed on graphics that con- vey messages in the form of jokes, sarcasm, etc. In the current era of the internet and social media, they are very quick to spread. If used as a part of a disinformation campaign, it can be quite tricky to notice the agenda behind them and has the potential to influence a large mass of people without them realizing it (Muller, 2018; Tard´aguila et al., 2018; Glowacki et al., 2018). To this end, SemEval 2021 Task 6 (Dimitrov et al., 2021) focuses on identifying such persuasive techniques (Miller, 1939) in a multimodal (visual- linguistic) setting. It consists of three subtasks that enable the participants to study the problem in each modality. Only the English textual cues are used in tasks A and B, while the visual cues are also used in task C. Task B is a modification of task A which further requires predicting the spans for each identified technique as well. ∗The authors have contributed equally. Figure 1: Sample memes demonstrating the multi- modal setting Meme classification is a multimodal problem that often requires visual and textual cues to con- vey a message. Memes can convey very different meanings if either of the cues is removed. A few samples are shown in figure 1 to demonstrate the importance of visual and textual cues for classifica- tion. We experiment with BERT (Devlin et al., 2019) based unimodal models for tasks A and B. Since they are state-of-the-art models for natural lan- guage understanding, they are a good choice for understanding the complex propaganda techniques in texts. Transformers (Vaswani et al., 2017) have limited sequence length, which limits their perfor- mance on longer data, but in the case of memes, the textual content is also very limited. For task C, we experiment with Visual- Linguistic (VL) models like UNITER (Chen et al., 2020), VisualBERT (Li et al., 2019), LXMERT (Tan and Bansal, …

Content truncated to 500 words