HHS at SemEval-2023 Task 10: A Comparative Analysis of Sexism Detection Based on the RoBERTa Model

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 963–968 July 13-14, 2023 ©2023 Association for Computational Linguistics HHS at SemEval-2023 Task 10: A Comparative Analysis of Sexism Detection Based on the RoBERTa Model Yao Zhang Yunnan University 18314275485@163.com Liqing Wang* Yunnan University wlq@ynu.edu.cn Abstract This paper describes the methods and mod- els applied by our team HHS in SubTask-A of SemEval-2023 Task 10 about sexism detec- tion. In this task, we trained with the officially released data and analyzed the performance of five models, TextCNN, BERT, RoBERTa, XLNet, and Sup-SimCSE-RoBERTa. The ex- periments show that most of the models can achieve good results. Then, we tried data aug- mentation, model ensemble, dropout, and other operations on several of these models, and com- pared the results for analysis. In the end, the most effective approach that yielded the best re- sults on the test set involved the following steps: enhancing the sexist data using dropout, feed- ing it as input to the Sup-SimCSE-RoBERTa model, and providing the raw data as input to the XLNet model. Then, combining the outputs of the two methods led to even better results. This method yielded a Macro-F1 score of 0.823 in the final evaluation phase of the SubTask-A of the competition. 1 Introduction Sexism has long existed in society, causing various asymmetries and inequities, and harming some peo- ple. Given the enormity of data and the fast-paced flow of information on the internet, addressing sex- ism still poses a significant challenge (Kirk et al., 2023). Web technologies are rapidly evolving and automated detection is widely used for sexism, ex- cept that most tools only give generic categories without further explanation of what sexism is and do not improve the interpretability, trust and under- standing of the tools. In the past, computers were frequently used to conduct application tests for sexism by many peo- ple. Multiple social media platforms have been used to analyze cyberbullying detection using var- ious topics based on deep learning models and transfer learning (Agrawal and Awekar, 2018). A method has been proposed that can detect and an- alyze three characteristics of Twitter tweets: pa- triarchal behavior, misogyny and sexism (Frenda et al., 2019). (Farrell et al., 2019) demonstrated an increasing pattern of misogynistic content and users as well as violent attitudes. Fine-grained, multi-label sexism account classification has been explored, and the first semi-supervised multi-label sexism account classification has been proposed (Abburi et al., 2021). (Guest et al., 2021) proposed a new hierarchical taxonomy for online misogyny and produced a related labeled dataset. Sexism takes many forms on the web, some are fixed words of sexism, some are new words, and many forms of sexism require contextual, temporal and geographi- cal understanding. Thus much of this sexism detec- tion is subjective and relative. This also means that sexism data is hard to find, requiring a lot of man- ual annotation effort. Therefore, we participated in the SemEval-2023 shared Task 10 (Kirk et al., 2023), which aims to identify …

Content truncated to 500 words