HateU at SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 581 - 584 July 14-15, 2022 ©2022 Association for Computational Linguistics HateU at SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification Aymé Arango, Jesus Perez-Martin, Arniel Labrada Millennium Institute for Foundational on Data, Chile University of Chile, Chile {aarango,jperez,alabrada}@dcc.uchile.cl Abstract Hate speech expressions in social media are not limited to textual messages; they can ap- pear in videos, images, or multimodal formats like memes. Existing work towards detecting such expressions has been conducted almost exclusively over textual content, and the analy- sis of pictures and videos has been very scarce. This paper describes our team proposal in the Multimedia Automatic Misogyny Identification (MAMI) task at SemEval 2022. The challenge consisted of identifying misogynous memes from a dataset where images and text transcrip- tions were provided. We reported a 71% of F-score using a multimodal system based on the CLIP model. 1 Introduction Expressions of hate are common in online envi- ronments, and they can appear in different types of multimedia content (Bhattacharya et al., 2020). However, the related work on hate-speech and of- fensive language detection is primarily focused on textual English content (Agrawal and Awekar, 2018; Hosseinmardi et al., 2015). But, even for the English language, the task is still not solved. Evidence of that is recent reports of the increasing amount of hateful content in social media1 follow- ing the occurrence of social or political events. Re- cent events like the COVID pandemic have brought a new wave of hate (Vishwamitra et al., 2020), with new targets and expressions including hateful memes (Pramanick et al., 2021). Therefore, the techniques for hate speech detection need to evolve towards new types of hate, representations, and languages. The lack of generality of existing resources along with the emergence of new nets of hate makes cur- rent systems quickly outdated2. 1https://www.channel4.com/news/george-floyd-death- has-led-to-increasing-online-hate-speech-report-claims 2https://whatsnewinpublishing.com/the-rise-of-hate- speech-and-what-the-media-can-do-about-it/ Most of the available datasets contain tweets (Waseem, 2016; Basile et al., 2019), Facebook and Youtube comments (Bosco et al., 2018) and, in general, textual content. Similar to The Hateful Memes Challenge 3 hosted by Facebook in 2020, The Multimedia Automatic Misogyny Identifica- tion (MAMI) challenge (Elisabetta Fersini, 2022) is an excellent opportunity for covering the hate speech detection task beyond written expressions. In this competition, the organizers provided a training set of 10,000 memes labeled as hate speech in two different forms: binary (misogynous, not misogynous) and multi-class (stereotype, shaming, objectification, and violence). The competition comprises two tasks: Task A, for binary identifi- cation of misogyny, and Task B, for fine-grained classification of misogynous memes. For final sys- tem evaluation, the organizers published a set of 1000 extra unlabeled memes. Each meme in train- ing and testing sets consists of an image with an overlay text. Each object in the dataset consists of an image and a transcription of the overlay text. This paper describes our team participation in Task A of the MAMI challenge. We encoded im- ages and texts using a pre-trained multi-modal model based …

Content truncated to 500 words