AMS_ADRN at SemEval-2022 Task 5: A Suitable Image-text Multimodal Joint Modeling Method for Multi-task Misogyny Identification

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 711 - 717 July 14-15, 2022 ©2022 Association for Computational Linguistics AMS_ADRN at SemEval-2022 Task 5: A Suitable Image-text Multimodal Joint Modeling Method for Multi-task Misogyny Identification Da Li Tencent. Inc Beijing, China danielsli@tencent.com Ming Yi Tencent. Inc Beijing, China mingyyi@tencent.com Yukai He Tencent. Inc Beijing, China yukaihe@tencent.com Abstract Women are influential online, especially in image-based social media such as Twitter and Instagram. However, many in the network en- vironment contain gender discrimination and aggressive information, which magnify gen- der stereotypes and gender inequality. There- fore, the filtering of illegal content such as gen- der discrimination is essential to maintain a healthy social network environment. In this paper, we describe the system developed by our team for SemEval-2022 Task 5: Multime- dia Automatic Misogyny Identification. More specifically, we introduce two novel systems to analyze these posts: a multimodal multi-task learning architecture that combines Bertweet for text encoding with ResNet-18 (He et al., 2016) for image representation, and a single- flow transformer structure which combines text embeddings from BERT-Embeddings and im- age embeddings from several different modules such as EfficientNet (Tan and Le, 2019) and ResNet (He et al., 2016). In this manner, we show that the information behind them can be properly revealed. Our approach achieves good performance on each of the two subtasks of the current competition, ranking 12th for Subtask A (0.746 macro F1-score), 10th for Subtask B (0.706 macro F1-score) while exceeding the official baseline results by high margins. 1 Introduction Women are influential online, especially in image- based social media such as Twitter and Instagram : 78 percent of women use social media mul- tiple times a day, compared with 65 percent of men.(Elisabetta Fersini) However, although the In- ternet has opened new opportunities for women, systematic offline inequality and discrimination are replicated in cyberspace in the form of offensive content against them. The popular communication tool in social media platforms is MEME. Memetics is essentially an image, which is characterized by the content of the image and the poster cover text introduced by human beings. Its main goal is fun or irony. Although most of them were created for a joke, in a very short period of time, people began to use them as a form of hatred against women, and then magnified gender stereotypes and gender inequality. Women are exposed to gender discrimi- nation and aggressive information in the network environment. Therefore, the filtering of illegal con- tent such as gender discrimination is essential to maintain a healthy social network environment. In this paper, we describe the system developed by our team for SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification. More specifi- cally, we introduce two novel systems to analyze these posts: a multimodal multi-task learning ar- chitecture that combines Bertweet (Nguyen et al., 2020) for text encoding with ResNet-18 (He et al., 2016) for image representation, and a single-flow transformer structure which combines text embed- dings from BERT-Embeddings and image embed- dings …

Content truncated to 500 words