SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 533 - 549 July 14-15, 2022 ©2022 Association for Computational Linguistics SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene University of Milano-Bicocca Viale Sarca 336, 20126, Milan, Italy elisabetta.fersini@unimib.it, francesca.gasparini@unimib.it g.rizzi10@campus.unimib.it, a.saibene2@campus.unimib.it Berta Chulvi, Paolo Rosso Universitat Politècnica de València Camino de Vera, Valencia, Spain berta.chulvi@upv.es prosso@dsic.upv.es Alyssa Lees, Jeffrey Sorensen Google Jigsaw, 111 8th Ave, New York, NY alyssalees@google.com sorenj@google.com Abstract The paper describes the SemEval-2022 Task 5: Multimedia Automatic Misogyny Identifica- tion (MAMI),which explores the detection of misogynous memes on the web by taking ad- vantage of available texts and images. The task has been organised in two related sub-tasks: the first one is focused on recognising whether a meme is misogynous or not (Sub-task A), while the second one is devoted to recognising types of misogyny (Sub-task B). MAMI has been one of the most popular tasks at SemEval-2022 with more than 400 participants, 65 teams in- volved in Sub-task A and 41 in Sub-task B from 13 countries. The MAMI challenge received 4214 submitted runs (of which 166 uploaded on the leader-board), denoting an enthusiastic participation for the proposed problem. The collection and annotation is described for the task dataset. The paper provides an overview of the systems proposed for the challenge, re- ports the results achieved in both sub-tasks and outlines a description of the main errors for a comprehension of the systems capabilities and for detailing future research perspectives. 1 Introduction Women have a strong presence online, particularly in image-based social media such as Twitter and In- stagram: 78% of women use social media multiple times per day compared to 65% of men (Depart- ment, 2019). However, while new opportunities for women have been opened on the web, systematic inequality and discrimination offline is replicated in online spaces in the form of offensive contents against them (Frenda et al., 2019; Anzovino et al., 2018; Farrell et al., 2019; Plaza-Del-Arco et al., 2020; Gasparini et al., 2018). A popular commu- nication tool in social media platforms are image macros popularly connoted as "memes" (Shifman, 2013). An internet meme is usually an image com- municating pictorial content with an overlaid text that is added a posteriori by the meme author, with the main goal of being funny and/or ironic (Shifman, 2013). Although many memes are cre- ated with humorous intent, others have political or activist ambitions. Few familiar with the for- mat would be surprised to learn that memes can be used to express hate against women, via sexist and aggressive messages in online environments (Paciello et al., 2021) that subsequently amplify the sexual stereotyping and gender inequality of the of- fline world (Franks, 2011). In order to counter this phenomenon, the Multimedia Automatic Misogyny Identification (MAMI) shared task has been organ- ised at SemEval-2022 (Emerson et al., 2022). The proposed challenge consists of the identification of misogynous memes, taking advantage of both text and images available as …

Content truncated to 500 words