Stanford MLab at SemEval-2023 Task 10: Exploring GloVe- and Transformer-Based Methods for the Explainable Detection of Online Sexism

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1791–1797 July 13-14, 2023 ©2023 Association for Computational Linguistics Stanford MLab at SemEval-2023 Task 10: Exploring GloVe- and Transformer-Based Methods for the Explainable Detection of Online Sexism Hee Jung Choi*, Trevor Chow*, Aaron Wan*, Hong Meng Yam*, Swetha Yogeswaran*, Beining Zhou* Stanford University {cheejung, tmychow, aaronwan, hongmeng, swethay, cathyzbn}@stanford.edu Abstract In this paper, we discuss the methods we applied at SemEval-2023 Task 10: Towards the Explainable Detection of Online Sexism. Given an input text, we perform three clas- sification tasks to predict whether the text is sexist and classify the sexist text into subcat- egories in order to provide an additional ex- planation as to why the text is sexist. We explored many different types of models, in- cluding GloVe embeddings as the baseline ap- proach, transformer-based deep learning mod- els like BERT, RoBERTa, and DeBERTa, en- semble models, and model blending. We ex- plored various data cleaning and augmentation methods to improve model performance. Pre- training transformer models yielded significant improvements in performance, and ensembles and blending slightly improved robustness in the F1 score. 1 Introduction Online sexism has the potential to inflict significant harm on women (Ortiz, 2023), and it is a serious issue that must be addressed. With the increasing prevalence of social media, it has become easy for groups of people to spread sexist ideas and threaten the safety of others, with online social networks be- coming increasingly inundated by sexist comments (Founta et al., 2018). There have been numerous previous works on the detection of online sexism as a whole (Schütz et al., 2021; Aldana-Bobadilla et al., 2021; Rodríguez-Sánchez et al., 2020), including even applying these models to non-English datasets (Ku- mar et al., 2021; de Paula et al., 2021; Jiang et al., 2021). However, almost all of these models do not focus on precisely classifying why a certain text conveys sexist sentiments, and instead provide a binary classification for whether the text is sexist. *Authors are listed alphabetically. All authors contributed equally to this work. However, this is the task which is most likely to make these models useful for content moderation, since they provide both the moderator and the plat- form’s users with a consistent explanation for why something was classified as sexist or not. In many cases, the difficulty of using machine classification when doing content moderation is the perception of it being decided by an arbitrary black box. While the models may empirically have a high accuracy, due to the high social sensitivity of this task, human intervention is almost always required. With laws such as the General Data Protection Reg- ulation (GDPR) established in Europe, which es- tablishes a "right to explanation"(Hoofnagle et al., 2019), there is thus a huge need for model explain- ability on top of existing performance optimization (Mathew et al., 2020). With more detailed feed- back about the categories of sexism, moderators can efficiently mitigate sexist sentiment online in a robust and rules-based manner, …

Content truncated to 500 words