PAIC at SemEval-2022 Task 5: Multi-Modal Misogynous Detection in MEMES with Multi-Task Learning And Multi-model Fusion

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 555 - 562 July 14-15, 2022 ©2022 Association for Computational Linguistics PAIC at SemEval-2022 Task 5: Multi-Modal Misogynous Detection in MEMES with Multi-Task Learning And Multi-model Fusion Meizhi Jin and Mengyuan Zhou and Mengfei Yuan and Dou Hu and Xiyang Du and Lianxin Jiang and Yang Mo and XiaoFeng Shi Ping An Life Insurance Company of China, Ltd. {JINMEIZHI005, ZHOUMENGYUAN425, YUANMENGFEI854, HUDOU470, DUXIYANG037, JIANGLIANXIN769, MOYANG853, SHIXIAOFENG309} @pingan.com.cn Abstract This paper describes our system used in the SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification (MAMI) . Multimedia automatic misogyny recognition consists of the identification of misogynous memes, taking ad- vantage of both text and images as sources of information. The task will be organized around two main subtasks: Task A is a binary classi- fication task, which should be identified either as misogynous or not misogynous. Task B is a multi-label classification task, in which the types of misogyny should be identified in poten- tial overlapping categories, such as stereotype, shaming, objectification, and violence. In this paper, we proposed a system based on multi- task learning for multi-modal misogynous de- tection in memes. Our system combined image features with text features to train a multi-label classification. The prediction results were ob- tained by the simple weighted average method of the results with different fusion models, and the results of Task A were corrected by Task B. Our system achieves a test accuracy of 0.755 on Task A (ranking 3rd on the final leaderboard) and the accuracy of 0.731 on Task B (ranking 1st on the final leaderboard). 1 Introduction In the era of mass online communication, more and more people like to share their thoughts on social media platforms. Social media platforms provide users with the ability to express themselves freely.However, it has also led to a rise in cyber hate, such as bullying, sarcasm, and misogyny, on the internet. A study has shown that women have a strong presence online, particularly in image-based social media such as Twitter and Instagram: 78% of women use social media multiple times per day compared to 65% of men. The Web has opened up a whole new world of opportunities for women, but systematic inequality and discrimination in the real world are also found in online spaces in the form of offensive content against them. Now that we live in a world where everything is connected through social media, hate speech against women used to be limited to a specific place or time. There’s no need to wait for a specific time or place. All you have to do is type a few keys on the keyboard. Because people on big social media sites make millions of posts every day, it is not possible to manually check all misogynistic posts. To help human curators, it is important to make algorithms that can tell when users post inappropriate content. Memes are one of the most popular ways to com- municate on social media platforms. …

Content truncated to 500 words