AdamR at SemEval-2023 Task 10: Solving the Class Imbalance Problem in Sexism Detection with Ensemble Learning

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1371–1381 July 13-14, 2023 ©2023 Association for Computational Linguistics AdamR at SemEval-2023 Task 10: Solving the Class Imbalance Problem in Sexism Detection with Ensemble Learning Adam Rydelek, Daryna Dementieva, and Georg Groh TU Munich, Department of Informatics, Germany {name.surname}@tum.de grohg@in.tum.de Abstract The Explainable Detection of Online Sexism task presents the problem of explainable sexism detection through fine-grained categorisation of sexist cases with three subtasks. Our team experimented with different ways to combat class imbalance throughout the tasks using data augmentation and loss alteration techniques. We tackled the challenge by utilising ensem- bles of Transformer models trained on different datasets, which are tested to find the balance between performance and interpretability. This solution ranked us in the top 40% of teams for each of the tracks. 1 Introduction Sexism and other forms of harassment can be com- monly observed on the internet. It can not only discourage attacked groups from interacting on the internet but also pose a serious threat (Fox et al., 2015). Especially with the growing popularity of social media platforms, the need for automation in detecting such occurrences arises. Such tools already exist and are being constantly developed but still face multiple issues (Gongane et al., 2022). One of the present problems in such systems is of- ten the lack of reasoning and interpretation behind the prediction itself. Presenting an explanation why the system has predicted a text to be sexist enhances trust in such systems and enables them to be more widely used (Ferrario and Loi, 2022). The Explainable Detection of Online Sexism (EDOS) task presents a problem of detecting sexism in En- glish texts gathered from popular Social Media platforms: Reddit and Gab (Kirk et al., 2023). The detection problem is proceeded by two subtasks: firstly, classifying the sexist entries into one of four categories, and finally, marking it as one of the eleven fine-grained vectors. In this work, we propose systems to tackle each of the subtasks which share the architecture and data pre-processing methods. For each problem, an ensemble of Transformer-based models is built and fine-tuned on English texts concerning the subtask. Models within the ensemble are trained on different sets of data. The provided EDOS datasets have been extended by data points from various sources concerning past research on the topic. The texts have been pre-processed and normalized to fit the schema provided with the train data to enhance the results. The performance metric chosen for the task is the macro-averaged F1 score, which treats each class equally and promotes balanced classification. Our contributions consist of providing ensemble- based solutions to all subtasks ranking us in the top 40% for each of them—36th place for Sexism De- tection, 29th for Category of Sexism, and 25th for Fine-grained Vector of Sexism tracks. We also test different methods of data augmentation and archi- tecture designs to tackle class imbalance problems. A comparison of different ensembles is presented, weighting its advantages and disadvantages over …

Content truncated to 500 words