QiNiAn at SemEval-2022 Task 5: Multi-Modal Misogyny Detection and Classification

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 736 - 741 July 14-15, 2022 ©2022 Association for Computational Linguistics QiNiAn at SemEval-2022 Task 5: Multi-Modal Misogyny Detection and Classification Qin Gu, Nino Meisinger, Anna-Katharina Dick University of Tuebingen {firstName.lastName}@student.uni-tuebingen.de Abstract In this paper, we describe our submission to the misogyny classification challenge at SemEval-2022. We propose two models for the two subtasks of the challenge: The first uses joint image and text classification to clas- sify memes as either misogynistic or not. This model uses a majority voting ensemble struc- ture built on traditional classifiers and addi- tional image information such as age, gender and nudity estimations. The second model uses a RoBERTa classifier on the text tran- scriptions to additionally identify the type of problematic ideas the memes perpetuate. Our submissions perform above all organizer sub- mitted baselines. For binary misogyny clas- sification, our system achieved the fifth place on the leaderboard, with a macro F1-score of 0.665. For multi-label classification identify- ing the type of misogyny, our model achieved place 19 on the leaderboard, with a weighted F1-score of 0.637. 1 Introduction Even though women are as much present online as men, some online spaces, such as microblogging websites, are still male dominated. Misogynistic jokes and memes are inevitably somewhat com- mon in certain parts of the Internet and have the potential to perpetuate harmful ideas about gender or instill false ideas and expectations. Notably, the way they are spread is often through various modal- ities, most commonly visual and textual. Therefore, it would be useful to have a system that could au- tomatically detect if certain combinations of texts and images are misogynistic or not. This is not a trivial task however, since misogyny can manifest in many different forms such as stereotyping, objec- tifying or threatening violence against women. A crucial difficulty in this multi-modal classification task is also the interplay between text and image. Some memes may appear harmless if only either the image or text are viewed separately. The Multi- media Automatic Misogyny Identification (MAMI) challenge at SemEval-2022 (Fersini et al., 2022) seeks to find solutions to solve this task. We propose two models that automatically detect English misogynistic memes and classify the type of problematic ideas they perpetuate.1 For simple binary misogyny detection, we created an ensemble model that makes predictions based on majority voting on two text-based and two image- based classifiers using partly hand-crafted features such as age, gender and nudity classification. For task B we relied only on text information and used a transformer based approach, creating a RoBERTa model (Liu et al., 2019) that classifies the type of misogyny that is perpetuated in the memes. 2 Background Automatically identifying misogynistic texts has been explored in the past. In 2020, this task was proposed as an EVALITA shared task, using Italian tweets (Fersini et al., 2022). So far, the research in the area of misogyny detection has mostly focused on pure text data from social media, specifically …

Content truncated to 500 words