Attention at SemEval-2023 Task 10: Explainable Detection of Online Sexism (EDOS)

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1972–1977 July 13-14, 2023 ©2023 Association for Computational Linguistics Attention at SemEval-2023 Task 10: Explainable Detection of Online Sexism (EDOS) Debashish Roy, Manish Shrivastava IIIT Hyderabad, India debashish.roy@students.iiit.ac.in m.shrivastava@iiit.ac.in Abstract In this paper, we have worked on explainability and understanding of the decisions made by models in the form of classification tasks. The task is divided into 3 subtasks. The first task consists of determining Binary Sexism Detec- tion. The second task describes the Category of Sexism. The third task describes a more Fine-grained Category of Sexism. Our work explores solving these tasks as a classification problem by fine-tuning transformer-based ar- chitecture. We have performed several experi- ments with our architecture, including combin- ing multiple transformers, using domain adap- tive pretraining on the unlabelled dataset pro- vided by Reddit and Gab, Joint learning, and taking different layers of transformers as input to a classification head. Our system (with team name ‘Attention’) was able to achieve a macro F1 score of 0.839 for task A, 0.5835 macro F1 score for task B and 0.3356 macro F1 score for task C at the Codalab SemEval Competition. Later we improved the accuracy of Task B to 0.6228 and Task C to 0.3693 in the test set. 1 Introduction Online sexism is a growing problem in this era, considering the large scale of online texts. Detect- ing online Sexism is an important NLP task, and has a lot of social impacts related to it. Sexism is defined as any abuse or negative sentiment that is directed towards women based on their gender or based on their gender combined with one or more other identity attributes (e.g. black women, trans women). There are some prior works regarding the detec- tion of sexist text (Rodríguez-Sánchez et al., 2022) at SemEval. Flagging what is sexist content and also explaining why it is sexist is very important aspects. We tried to focus on interpretability, trust, and understanding of the decisions made by mod- els. Having an automated system that is capable of understanding, interpreting, and classifying sexist text will be a major step to make online sites safe. Explainable Detection of Online Sexism (EDOS) at SemEval 2023 (Kirk et al., 2023), tries to solve this problem by exploring different systems that can categorize the dataset created with texts from Reddit and Gab. The problem is divided into 3 subtasks. The first task is formulated as Binary Sexism Detection, where the main idea is to detect whether a text is sexist or not. The second task describes the Category of Sex- ism. So, if the text is sexist, under which category it belongs? The categories are threats, derogation (treating someone as little worthy), animosity (a strong dislike or unfriendly feeling), and prejudiced discussions (an unfair feeling of dislike for a person or group because of race, sex, religion, etc). This makes this task a four-class classification problem given the text is sexist. The third task describes the …

Content truncated to 500 words