Poirot at SemEval-2022 Task 5: Leveraging Graph Network for Misogynistic Meme Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 793 - 801 July 14-15, 2022 ©2022 Association for Computational Linguistics Poirot at SemEval-2022 Task 5: Leveraging Graph Network for Misogynistic Meme Detection Harshvardhan Srivastava Oracle India Indian Institute of Technology, Kharagpur harshvardhan.srivastava@oracle.com Abstract In recent years, there has been an upsurge in a new form of entertainment medium called memes. These memes although seemingly in- nocuous have transcended the boundary of on- line harassment against women and created an unwanted bias against them. To help alleviate this problem, we propose an early fusion model for the prediction and identification of misog- ynistic memes and their type in this paper for which we participated in SemEval-2022 Task 5. The model receives as input meme image with its text transcription with a target vector. Given that a key challenge with this task is the combination of different modalities to pre- dict misogyny, our model relies on pre-trained contextual representations from different state- of-the-art transformer-based language models and pre-trained image pre-trained models to get an effective image representation. Our model achieved competitive results on both SubTask- A and SubTask-B with the other competing teams and significantly outperforms the base- lines. 1 Introduction Meme culture in today’s virtual climate gives us a variety of insight into the pop culture, general ide- ology and linguistic conversational manner of the generation. To understand the internet culture, it becomes essential to study memes (Shifman, 2013) and the impact it has on people on the internet. Some of the most popular communication tools on social media platforms are memes. Memes are essentially images characterized by the content of a picture overlaid with text that was introduced by people with the main purpose of being inter- esting and ironic. Women have a strong presence online, especially on image-based social media like Twitter, Snapchat, and Instagram. 78% of women use social media several times a day, compared to 65% of men (Fersini et al., 2022). While new opportunities are being opened up for women on- line, systematic inequality and discrimination are being replicated offline from these online spaces in the form of offensive content for women. Most of them were created to make funny jokes, but soon people began to use them as a form of hatred for women, leading to sexist and offensive messages in the online environment, and as a consequence, the sexual stereotyping and gender inequality of the offline world where sexuality stereotypes and gender inequality have been strengthened. This insensitive and obscene type of meme has a pro- found effect on a person’s mental health and can exhibit harmful effects on cognitive and emotional processes leading to mental illnesses as shown in Paciello et al. (2021). In this work, we present team Poirot’s solution to SemEval - 2022 Task 5 competition as described in detail in Fersini et al. (2022). We focused our efforts on our primary approach of building a Multi- Modal module that uses features from both images and text. Furthermore, in …

Content truncated to 500 words