Alexa at SemEval-2023 Task 10: Ensemble Modeling of DeBERTa and BERT Variations for Identifying Sexist Text

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1644–1649 July 13-14, 2023 ©2023 Association for Computational Linguistics Alexa at SemEval-2023 Task 10: Ensemble Modeling of DeBERTa and BERT Variations for Identifying Sexist Text Mutaz Younes Dep. of Computer Science Maharishi International University Iowa, USA mutazyounes@gmail.com Ali Kharabsheh Dep. of Computer Science Yarmouk University Irbid, Jordan alikharabsha12@gmail.com Mohammad Bani Younes Dep. of Computer Science Ajloun National University Irbid, Jordan aliyounes1962@gmail.com Abstract This study presents an ensemble approach for detecting sexist text in the context of the Semeval-2023 task 10. Our approach lever- ages 18 models, including DeBERTa-v3-base models with different input sequence lengths, a BERT-based model trained on identifying hate speech, and three more models pre-trained on the task’s unlabeled data with varying input lengths. The results of our framework on the de- velopment set show an f1-score of 84.92% and on the testing set 84.55%, effectively demon- strating the strength of the ensemble approach in getting accurate results. 1 Introduction Sexist language has been a persistent issue in vari- ous forms of communication, including online com- munication. The use of sexist language perpetuates gender stereotypes and reinforces the marginaliza- tion of certain groups, particularly women. Detect- ing and addressing instances of sexist language is crucial for promoting gender equality and reducing discrimination. In recent years, there has been a growing interest in developing methods for automatically detecting sexist language. The Semeval-2023 task 10 pro- vides (Kirk et al., 2023) a platform for researchers to explore and evaluate various approaches to iden- tifying sexist text. This task involves identifying instances of sexist language in various forms of on- line communication, including social media posts, comments, and reviews. Previous research has explored different meth- ods for identifying sexist language, including rule- based approaches, machine-learning techniques, and deep-learning models. While these approaches have shown promising results, the challenge of de- tecting sexist language remains complex due to the subtleties and nuances of language use. Our approach for identifying the sexist language in the Semeval-2023 task 10 subtask A involves an ensemble of 18 different models, including DeBERTa-v3-base and pre-trained BERT-based models pre-trained on the task’s unlabeled data. The model’s predictions are combined to improve overall performance. Our method achieved an f1- score of 84.92% on the development dataset and 84.55% on the testing dataset, ranking 30th out of 89 participating teams. Section 2 of this paper introduces an overview of the related work in the field of sexist text identi- fication. Section 3 gives an overview of the details of the dataset we used in this research. Section 4 presents key characteristics of the ensemble ap- proach and provides a detailed description of the models used. Finally, in Section 5, we conclude the paper and discuss future directions for research in this area. 2 Background Sexism against women on social media has become a growing concern, leading researchers to develop automatic systems that detect sexist text. Online competitions such as the SemEval-2022 Task 5 on "Multimedia automatic misogyny identification" …

Content truncated to 500 words