Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text Classification

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5010–5019 August 1–6, 2021. ©2021 Association for Computational Linguistics 5010 Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text Classification Xuepeng Wang, Li Zhao, Bing Liu, Tao Chen, Feng Zhang, Di Wang Tencent AI Platform Department, China {woodswang,lilythzhao,andybliu}@tencent.com {vitochen,jayzhang,diwang}@tencent.com Abstract Hierarchical Text Classification (HTC) is a challenging task that categorizes a textual de- scription within a taxonomic hierarchy. Most of the existing methods focus on modeling the text. Recently, researchers attempt to model the class representations with some resources (e.g., external dictionaries). However, the con- cept shared among classes which is a kind of domain-specific and fine-grained informa- tion has been ignored in previous work. In this paper, we propose a novel concept-based label embedding method that can explicitly represent the concept and model the sharing mechanism among classes for the hierarchi- cal text classification. Experimental results on two widely used datasets prove that the pro- posed model outperforms several state-of-the- art methods. We release our complementary resources (concepts and definitions of classes) for these two datasets to benefit the research on HTC. 1 Introduction Text classification is a classical Natural Language Processing (NLP) task. In the real world, the text classification is usually cast as a hierarchical text classification (HTC) problem, such as patent collec- tion (Tikk et al., 2005), web content collection (Du- mais and Chen, 2000) and medical record cod- ing (Cao et al., 2020). In these scenarios, the HTC task aims to categorize a textual description within a set of labels that are organized in a structured class hierarchy (Silla and Freitas, 2011). Lots of researchers devote their effort to investigate this challenging problem. They have proposed vari- ous HTC solutions, which are usually categorized into flat (Aly et al., 2019), local (Xu and Geng, 2019), global (Qiu et al., 2011) and combined ap- proaches (Wehrmann et al., 2018). In most of the previous HTC work, researchers mainly focus on modeling the text, the labels are Figure 1: Concepts shared among classes in WOS. simply represented as one-hot vectors (Zhu and Bain, 2017; Wehrmann et al., 2018). Actually, the one-hot vectors act as IDs without any semantic in- formation. How to describe a class is also worthy of discussion. There is some work that embeds labels into a vector space which contains more semantic information. Compared with one-hot representa- tions, label embeddings have advantages in captur- ing domain-specific information and importing ex- ternal knowledge. In the field of text classification (includes the HTC task), researchers propose sev- eral forms of label embeddings to encode different kinds of information, such as 1) anchor points (Du et al., 2019), 2) compatibility between labels and words (Wang et al., 2018; Huang et al., 2019; Tang et al., 2015), 3) taxonomic hierarchy (Cao et al., 2020; Zhou et al., 2020) and 4) external knowl- edge (Rivas Rojas et al., 2020). Although the external knowledge has been proven …

Content truncated to 500 words