NLP-LTU at SemEval-2023 Task 10: The Impact of Data Augmentation and Semi-Supervised Learning Techniques on Text Classification Performance on an Imbalanced Dataset

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1421–1427 July 13-14, 2023 ©2023 Association for Computational Linguistics NLP-LTU at SemEval-2023 Task 10: The Impact of Data Augmentation and Semi-Supervised Learning Techniques on Text Classification Performance on an Imbalanced Dataset Sana Al-Azzawi György Kovács Filip Nilsson Tosin Adewumi Marcus Liwicki EISLAB Machine Learning, Luleå University of Technology, 977 54 Luleå, Sweden firstname.lastname@ltu.se Abstract In this paper, we propose a methodology for task 10 of SemEval23, focusing on detecting and classifying online sexism in social me- dia posts. The task is tackling a serious is- sue, as detecting harmful content on social media platforms is crucial for mitigating the harm of these posts on users. Our solution for this task is based on an ensemble of fine- tuned transformer-based models (BERTweet, RoBERTa, and DeBERTa). To alleviate prob- lems related to class imbalance, and to improve the generalization capability of our model, we also experiment with data augmentation and semi-supervised learning. In particular, for data augmentation, we use back-translation, ei- ther on all classes, or on the underrepresented classes only. We analyze the impact of these strategies on the overall performance of the pipeline through extensive experiments. while for semi-supervised learning, we found that with a substantial amount of unlabelled, in- domain data available, semi-supervised learn- ing can enhance the performance of certain models. Our proposed method (for which the source code is available on Github12) attains an F1-score of 0.8613 for sub-taskA, which ranked us 10th in the competition. 1 Introduction Remarkable technological advancements have made it simpler for people from diverse back- grounds to interact through social media using posts and comments written in natural language. These opportunities, however, come with their own challenges. Hateful content on the Internet in- creased to such levels that manual moderation can- not possibly deal with it (Gongane et al., 2022). Thus, precise identification of harmful content on social media is vital for ensuring that such content can be discovered and dealt with, minimizing the 1github.com/SanaNGU/semeval23-task10-sexism-detection- 2huggingface.co/NLP-LTU/bertweet-large-sexism-detector risk of victim harm and making online platforms safer and more inclusive. Detecting online sexism on social media remains a challenge in natural language processing (NLP), and the Explainable Detection of Online Sexism (EDOS) shared task on SemEval23 (Kirk et al., 2023) addresses this problem. The task has three main sub-tasks: (i) task A; binary sexism detection, in which we determine whether a given sentence contains sexist content, (ii) task B; sexism classi- fication, which places sexist sentences into four categories: threats, derogation, animosity, and prej- udiced discussions, and (iii) task C; fine-grained vector of sexism, an eleven-class categorization for sexist posts in which systems must predict one of 11 fine-grained vectors. One major challenge of this task is the imbal- anced class distribution. For instance, sub-task A consists of only 3398 sexist posts, and 10602 non- sexist ones. Using an imbalanced dataset to train models can result in prediction bias towards the majority class (Johnson and Khoshgoftaar, 2019). In this paper, we (team …

Content truncated to 500 words