Hyperbolic Capsule Networks for Multi-Label Classification

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3115–3124 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3115 Hyperbolic Capsule Networks for Multi-Label Classification Boli Chen, Xin Huang, Lin Xiao, Liping Jing Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University, Beijing, China {18120345, 18120367, 17112079, lpjing}@bjtu.edu.cn Abstract Although deep neural networks are effective at extracting high-level features, classification methods usually encode an input into a vec- tor representation via simple feature aggrega- tion operations (e.g. pooling). Such opera- tions limit the performance. For instance, a multi-label document may contain several con- cepts. In this case, one vector can not suffi- ciently capture its salient and discriminative content. Thus, we propose Hyperbolic Cap- sule Networks (HYPERCAPS) for Multi-Label Classification (MLC), which have two mer- its. First, hyperbolic capsules are designed to capture fine-grained document information for each label, which has the ability to char- acterize complicated structures among labels and documents. Second, Hyperbolic Dynamic Routing (HDR) is introduced to aggregate hy- perbolic capsules in a label-aware manner, so that the label-level discriminative information can be preserved along the depth of neural net- works. To efficiently handle large-scale MLC datasets, we additionally present a new routing method to adaptively adjust the capsule num- ber during routing. Extensive experiments are conducted on four benchmark datasets. Com- pared with the state-of-the-art methods, HY- PERCAPS significantly improves the perfor- mance of MLC especially on tail labels. 1 Introduction The main difference between Multi-Class Clas- sification (MCC) and Multi-Label Classification (MLC) is that datasets in MCC have only serval mutually exclusive classes, while datasets in MLC contain much more correlated labels. MLC allows label co-occurrence in one document, which indi- cates that the labels are not disjointed. In addition, a large fraction of the labels are the infrequently occurring tail labels (Bhatia et al., 2015), which is also referred as the power-law label distribution. Figure 1: The power-law label distribution of EUR- LEX57K with Y-axis on log-scale. Division is based on average number of training instances. Figure 1 illustrates the label distribution of EUR- LEX57K (Chalkidis et al., 2019). A multi-label document usually has serval head and tail labels, and hence contain several concepts about both its head and tail labels simultaneously. Recent works for text classification, such as CNN-KIM (Kim, 2014) and FASTTEXT (Joulin et al., 2017), focus on encoding a document into a fixed-length vector as the distributed document representation (Le and Mikolov, 2014). These en- coding based deep learning methods use simple operations (e.g. pooling) to aggregate features ex- tracted by neural networks and construct the doc- ument vector representation. A Fully-Connected (FC) layer is usually applied upon the document vector to predict the probability of each label. And each row in its weight matrix can be interpreted as a label vector representation (Du et al., 2019b). In this way, the label probability can be predicted by computing the dot product between label and doc- ument vectors, which is proportional to the …

Content truncated to 500 words