LeCun at SemEval-2021 Task 6: Detecting Persuasion Techniques in Text Using Ensembled Pretrained Transformers and Data Augmentation
25439Authors
nan
Abstract
AbstractWe developed a system for task 6 sub-task 1 for detecting propaganda in memes. An external dataset and augmentation data-set were used to extend the official competition data-set. Data augmentation techniques were applied on the external data-set and competition data-set to come up with the augmented data-set. We trained 5 transformers (DeBERTa, and 4 RoBERTa) and ensembled them to make the prediction. We trained 1 RoBERTa model initially on the augmented data-set for a few epochs and then fine-tuned it on the competition data-set which improved the f1-micro up to 0.1 scores. After that, another initial RoBERTa model was trained on the external data-set merged with the augmented data-set for few epochs and fine-tuned it on the competition data-set. Furthermore, we ensembled the initial models with the models after fine-tuning. For the final model in the ensemble, we trained a DeBERTa model on the augmented data-set without fine-tuning it on the competition data-set. Finally, we averaged the output of each model in the ensemble to make the prediction.
Keywords
Full Content
Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1068–1074 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1068 LeCun at SemEval-2021 Task 6: Detecting Persuasion Techniques in Text Using Ensembled Pretrained Transformers and Data Augmentation Dia Abujaber* Ahmed Qarqaz* Malak Abdullah Jordan University of Science and Technology Irbid, Jordan daabujaber17, afalqarqaz17@cit.just.edu.jo mabdullah@just.edu.jo Abstract This paper presents one of the top systems for the SemEval-2021 task 6 (Dimitrov et al., 2021), “detection of persuasion techniques in text and images”. The proposed system, Le- Cun, targets subtask-1 for detecting propa- ganda techniques based on the textual con- tent of a meme. We have used an exter- nal dataset from a previous relevant SemEval competition (Martino et al., 2020). We also have articulated another dataset using data- augmentation techniques. The final proposed model consisted of 5 ensemble transformers (four RoBERTa models and one DeBERTa), each trained on either a different dataset or pre- processing. Apparently, ensembling models trained on different datasets improve perfor- mance more than ensembling models trained on the same dataset/preprocessing. Also, it is obvious, fine-tuning the model on the Compe- tition dataset after training it for a few epochs on the other datasets would improve the f1- micro up to 0.1 additional scores. The final model achieved an f1-micro score of 0.512 on the test dataset and an f1-micro of 0.647 on the development dataset. 1 Introduction The definition of Memes was constantly changing since it was first conceived. But, Memes eventu- ally got an academic definition, called an “Internet Meme”. As Davison (2012) Internet Meme can roughly be defined as “a piece of culture, typically a joke, which gains influence through online trans- mission”. But what makes Internet memes unique is the speed of their transmission and the fidelity of their form. Therefore the Internet meme can act as a powerful medium for persuasion techniques that preach an ideology or way of thinking. (Moody- Ramirez and Church, 2019) On the other hand, the term ”propaganda” is de- fined as a form of communication that employs persuasive strategies and attempts to achieve a re- sponse that furthers the desired intent of the pro- pagandist (Jowett and O’donnell, 2018). With the rise of social media, a new form of propaganda rises called “Computational Propaganda.” The au- thor in (Woolley and Howard, 2017) defined Com- putational Propaganda as “The use of algorithms, automation, and human curation to purposefully distribute misleading information over social me- dia networks”. Task 6 at SemEval-2021 (Dimitrov et al., 2021), detection of persuasion techniques in text and im- ages, defined three subtasks. The first two subtasks deal with the textual contents of memes that ask the participants to identify which of the 20 propaganda techniques are in the text. While the third subtask encourages the participants to determine which of the 22 techniques are in the meme’s textual and visual content. This paper proposes a solution for subtask1 that uses pre-trained language models to detect propaganda and possibly even identify the persuasion …
Content truncated to 500 words
Labeled Text
AbstractWe developed a system for task 6 sub-task 1 for detecting propaganda in memes. An external dataset and augmentation data-set were used to extend the official competition data-set. Data augmentation techniques were applied on the external data-set and competition data-set to come up with the augmented data-set. We trained 5 transformers (DeBERTa, and 4 RoBERTa) and ensembled them to make the prediction. We trained 1 RoBERTa model initially on the augmented data-set for a few epochs and then fine-tuned it on the competition data-set which improved the f1-micro up to 0.1 scores. After that, another initial RoBERTa model was trained on the external data-set merged with the augmented data-set for few epochs and fine-tuned it on the competition data-set. Furthermore, we ensembled the initial models with the models after fine-tuning. For the final model in the ensemble, we trained a DeBERTa model on the augmented data-set without fine-tuning it on the competition data-set. Finally, we averaged the output of each model in the ensemble to make the prediction.
Publication Details
- Paper ID:
- 25439
- Year:
- 2021
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan