DCU at SemEval-2023 Task 10: A Comparative Analysis of Encoder-only and Decoder-only Language Models with Insights into Interpretability

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1736–1750 July 13-14, 2023 ©2023 Association for Computational Linguistics DCU at SemEval-2023 Task 10: A Comparative Analysis of Encoder-only and Decoder-only Language Models with Insights into Interpretability Kanishk Verma and Kolawole Adebayo and Joachim Wagner and Brian Davis ADAPT Centre, School of Computing, Dublin City University, Dublin Ireland {firstname.lastname}@adaptcentre.ie Abstract We conduct a comparison of pre-trained encoder-only and decoder-only language mod- els with and without continued pre-training, to detect online sexism. Our fine-tuning-based classifier system achieved the 16th rank in the SemEval 2023 Shared Task 10 Subtask A that asks to distinguish sexist and non-sexist texts. Additionally, we conduct experiments aimed at enhancing the interpretability of systems de- signed to detect online sexism. Our findings provide insights into the features and decision- making processes underlying our classifier sys- tem, thereby contributing to a broader effort to develop explainable AI models to detect online sexism. 1 Introduction Sexism represents “assumptions, beliefs, theories, stereotypes, and broader cultural narratives that represent men and women as importantly differ- ent” (Manne, 2017). Sexist beliefs construct and strengthen narratives that normalise gender distinc- tions between males and females by promoting the idea of innate variances between the two genders. According to feminist studies (Drakett et al., 2018; Andreasen, 2021), occurrences of online sexism and harassment are frequently depicted as “permis- sible” by presenting them as a kind of amusement or joke. Challenging these narratives and working to eliminate gender-based discrimination is crucial to creating a more just and equitable society, espe- cially over the internet. The shared task Explainable Detection of Online Sexism (EDOS) was part of SemEval 2023 and there were three subtasks associated with it (Kirk et al., 2023). This paper describes the DCU system submitted for all the three subtasks of the EDOS shared task. Subtask A The first subtask is a straightforward binary classification problem, in which the system has to classify a textual phrase or a sentence as sexist or not sexist. Subtask B The second subtask is a mutli-class classification problem, in which the system has to classify a social media post in one of four different categories: (a) threats, (b) derogation, (c) animosity and (d) prejudiced discussions. Subtask C The third subtask is a fine-grained classification problem, in which the system has to classify the sexist textual phrase or sentence in eleven sub-types of the earlier mentioned four categories of Subtask B. Sections 3 and 4 describe our system in detail. The results of our system are discussed in Section 5. Furthermore, since the shared-task pertains to “ex- plainable” detection of online sexism, we have made an extra effort contributing towards the in- terpretability of classification systems. To achieve this objective, we recruited two gender studies and one online harm research scholars to annotate ten randomly selected phrases that our submitted sys- tem failed to accurately classify as “sexist” and “not sexist”. These contributions are discussed in Section 6. 2 Background We build on fine-tuning pre-trained language …

Content truncated to 500 words