Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distribution

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8153–8161 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8153 Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distribution Yi Huang Data and Analytics Chapter Roche (China) Holding Ltd., China yi.huang.yh4@roche.com Buse Giledereli Computer Engineering Bogazici University, Turkey Data and Analytics Chapter Roche Müstahzarları San A.¸S., Turkey busegiledereli@gmail.com Abdullatif Köksal Computer Engineering Bogazici University, Turkey abdullatif.koksal@boun.edu.tr Arzucan Özgür Computer Engineering Bogazici University, Turkey arzucan.ozgur@boun.edu.tr Elif Ozkirimli Data and Analytics Chapter F. Hoffmann-La Roche AG Switzerland elif.ozkirimli@roche.com Abstract Multi-label text classification is a challenging task because it requires capturing label de- pendencies. It becomes even more challeng- ing when class distribution is long-tailed. Re- sampling and re-weighting are common ap- proaches used for addressing the class imbal- ance problem, however, they are not effective when there is label dependency besides class imbalance because they result in oversampling of common labels. Here, we introduce the ap- plication of balancing loss functions for multi- label text classification. We perform experi- ments on a general domain dataset with 90 labels (Reuters-21578) and a domain-specific dataset from PubMed with 18211 labels. We find that a distribution-balanced loss function, which inherently addresses both the class im- balance and label linkage problems, outper- forms commonly used loss functions. Dis- tribution balancing methods have been suc- cessfully used in the image recognition field. Here, we show their effectiveness in natural language processing. Source code is avail- able at https://github.com/blessu/ BalancedLossNLP. 1 Introduction Multi-label text classification is one of the core topics in natural language processing (NLP) and is used in many applications such as search (Prabhu et al., 2018) and product categorization (Agrawal et al., 2013). It aims to find the related labels from a fixed-set of labels for a given text that may have multiple labels. Figure 1 demonstrates examples Title 1 PENN CENTRAL <PC> SELLS U.K. UNIT Labels 1 acq (1650), strategic-metal (16) nickel (8) Title 2 U.S. MINT SEEKING OFFERS ON COPPER, NICKEL Labels 2 copper (47), nickel (8) Figure 1: Samples for multi-label text classification task from Reuters-21578 dataset. Only titles are shown for illustration. Numbers after the labels indicate total number of occurrences in the dataset. from the Reuters-21578 multi-label text classifica- tion dataset (Hayes and Weinstein, 1990). Here, for the document with the title PENN CENTRAL <PC> SELLS U.K. UNIT, the aim is to find the la- bels acq (acquisitions), strategic-metal, and nickel from 90 labels. Multi-label classification becomes complicated when there is a long-tailed distribution (class imbal- ance) and linkage (co-occurrence) of labels. Class imbalance occurs when a small subset of the labels (namely head labels) have many instances, while majority of the labels (namely tail labels) have only a few instances. For example, half of the labels in the Reuters dataset, including copper, strategic- metal, and nickel, occur in less than 5% of the training data. Label co-occurrence or label linkage is a challenge when some head labels co-occur with rare or tail labels, resulting in …

Content truncated to 500 words