UMUTeam at SemEval-2022 Task 5: Combining image and textual embeddings for multi-modal automatic misogyny identification

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 742 - 747 July 14-15, 2022 ©2022 Association for Computational Linguistics UMUTeam at SemEval-2022 Task 5: Combining image and textual embeddings for multi-modal automatic misogyny identification José Antonio García-Díaz Facultad de Informática, Universidad de Murcia, Campus de Espinardo, 30100, Spain joseantonio.garcia8@um.es Camilo Caparros-Laiz Facultad de Informática, Universidad de Murcia, Campus de Espinardo, 30100, Spain camilo.caparrosl@um.es Rafael Valencia-García Facultad de Informática, Universidad de Murcia, Campus de Espinardo, 30100, Spain valencia@um.es Abstract In this manuscript we describe the participa- tion of the UMUTeam on the MAMI shared task proposed at SemEval 2022. This task is concerning the identification of misogynous content from a multi-modal perspective. Our participation is grounded on the combination of different feature sets within the same neural network. Specifically, we combine linguistic features with contextual transformers based on text (BERT) and images (BEiT). Besides, we also evaluate other ensemble learning strategies and the usage of non-contextual pretrained em- beddings. Although our results are limited, we outperform all the baselines proposed, achiev- ing position 36 in the binary classification task with a macro F1-score of 0.687, and position 28 in the multi-label task of misogynous cate- gorisation, with an macro F1-score of 0.663. 1 Introduction This manuscript describes the participation of the UMUTeam in the Multimedia Automatic Misogyny Identification (MAMI) shared task (Fersini et al., 2022), proposed at SemEval 2022. This shared- task consists in the identification and categorisation of misogynous content from a dataset composed of memes (Dawkins and Davis, 2017). A meme is essentially a pictorial content with an overlay- ing text that pretends to be funny. However, some of these memes are being used as a form of hate against women, with sexist messages in online so- cial networks that amplify misogynous traits such as sexual stereotyping or gender inequality. MAMI shared task proposes two challenges. A binary classification task, in which each meme should be labelled as misogynous or not misog- ynous, and a multi-label classification task, to cate- gorise different misogynous traits, namely shaming, stereotype, objectification, and violence. 2 Background information From the last years, the number of shared tasks in workshops regarding hate-speech and misog- yny detection are increasing. To name just but a few, in Italian there is the EVALITA 2018 dataset (Bosco et al., 2018); in Spanish, the AMI 2018 dataset (Fersini et al., 2018) and the EXIST dataset (Rodríguez-Sánchez et al., 2021). In German, the GermEval 2021 (Risch et al., 2021) dataset. The common approaches for misogyny detection and categorisation consists in the training of an au- tomatic machine learning classifier. For example, the authors of (Anzovino et al., 2018) compiled and labelled a corpus from Twitter focused on misog- ynous content, and evaluate several feature sets and machine-learning models. In Spanish, a simi- lar approach was conducted in (García-Díaz et al., 2021), in which the authors released the Spanish MisoCorpus 2020. This dataset is organised into three splits: (1) VARW (Violence Against Relevant Women), focused on aggressive messages on …

Content truncated to 500 words