A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 523–540 May 2-6, 2023 ©2023 Association for Computational Linguistics A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing Sophie Henning1,2 William Beluch1 Alexander Fraser2 Annemarie Friedrich1 1 Bosch Center for Artificial Intelligence, Renningen, Germany 2 Center for Information and Language Processing, LMU Munich, Germany sophieelisabeth.henning|william.beluch@de.bosch.com fraser@cis.lmu.de annemarie.friedrich@de.bosch.com Abstract Many natural language processing (NLP) tasks are naturally imbalanced, as some target cate- gories occur much more frequently than others in the real world. In such scenarios, current NLP models tend to perform poorly on less frequent classes. Addressing class imbalance in NLP is an active research topic, yet, finding a good approach for a particular task and im- balance scenario is difficult. In this survey, the first overview on class im- balance in deep-learning based NLP, we first discuss various types of controlled and real- world class imbalance. Our survey then covers approaches that have been explicitly proposed for class-imbalanced NLP tasks or, originating in the computer vision community, have been evaluated on them. We organize the methods by whether they are based on sampling, data augmentation, choice of loss function, staged learning, or model design. Finally, we discuss open problems and how to move forward. 1 Introduction Class imbalance is a major problem in natural lan- guage processing (NLP), because target category distributions are almost always skewed in NLP tasks. As illustrated by Figure 1, this often leads to poor performance on minority classes. Which cate- gories matter is highly task-specific and may even depend on the intended downstream use. Develop- ing methods that improve model performance in imbalanced data settings has been an active area for decades (e.g., Bruzzone and Serpico, 1997; Japkow- icz et al., 2000; Estabrooks and Japkowicz, 2001; Park and Zhang, 2002; Tan, 2005), and is recently gaining momentum in the context of maturing neu- ral approaches (e.g., Buda et al., 2018; Kang et al., 2020; Li et al., 2020; Yang et al., 2020; Jiang et al., 2021; Spangher et al., 2021). The problem is exac- erbated when classes overlap in the feature space (Lin et al., 2019; Tian et al., 2020). For example, labels 0k 2.5k #train inst. 101 102 103 label count 0 1 test F1 (a) Single-label relation classification on TACRED (Zhou and Chen, 2021) labels 0k 1k #train inst. 101 102 103 CPC class train count 0 1 test F1 (b) Hierarchical multi-label patent classification (Pujari et al., 2021) labels 0k 2.5k #train inst. 0 1000 2000 3000 label count 0 1 test F1 (c) Implicit discourse rela- tion classification (PDTB) (Shi and Demberg, 2019) labels 0k 20k #train inst. 102 104 relation train count 0 1 dev F1 (d) UD dependency parsing using RoBERTa on EWT (Grünewald et al., 2021) Figure 1: Class imbalance has a negative effect on per- formance especially for minority classes in a variety of NLP tasks. Upper charts show label count distributions, lower part show …

Content truncated to 500 words