Joint Learning of Hyperbolic Label Embeddings for Hierarchical Multi-label Classification

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2829–2841 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2829 Joint Learning of Hyperbolic Label Embeddings for Hierarchical Multi-label Classification Soumya Chatterjee1∗ Ayush Maheshwari1∗ Ganesh Ramakrishnan1 Saketha Nath Jagarlapudi2 1 Indian Institute of Technology Bombay 2 Indian Institute of Technology Hyderabad {soumya, ayusham, ganesh}@cse.iitb.ac.in, saketha@iith.ac.in Abstract We consider the problem of multi-label clas- sification, where the labels lie in a hierarchy. However, unlike most existing works in hier- archical multi-label classification, we do not assume that the label-hierarchy is known. En- couraged by the recent success of hyperbolic embeddings in capturing hierarchical relations, we propose to jointly learn the classifier param- eters as well as the label embeddings. Such a joint learning is expected to provide a two- fold advantage: i) the classifier generalises bet- ter as it leverages the prior knowledge of ex- istence of a hierarchy over the labels, and ii) in addition to the label co-occurrence infor- mation, the label-embedding may benefit from the manifold structure of the input datapoints, leading to embeddings that are more faithful to the label hierarchy. We propose a novel for- mulation for the joint learning and empirically evaluate its efficacy. The results show that the joint learning improves over the baseline that employs label co-occurrence based pre-trained hyperbolic embeddings. Moreover, the pro- posed classifiers achieve state-of-the-art gen- eralization on standard benchmarks. We also present evaluation of the hyperbolic embed- dings obtained by joint learning and show that they represent the hierarchy more accurately than the other alternatives. The source code of the paper is available here. 1 Introduction The problem of multi-label text classification is well known and extensively studied in litera- ture (McCallum, 1999; Yang et al., 2009; Liu et al., 2017). The fundamental assumption is that a doc- ument is associated with multiple labels from a fixed vocabulary of labels. Often, these labels are organised in a hierarchical structure. For ex. con- sider a sample headline from the NYT (NewYork ∗Equal contribution Times) corpus “Voice Recognition Is Improving, but Don’t Stop the Elocution Lessons” for which labels are “Top/News/Technology”. Here, labels are arranged in a hierarchy, hereafter referred as label hierarchy. We undertake the task of labelling documents with classes that are hierarchically or- ganised; this problem is popularly known as hi- erarchical multi-label text classification (HMC). HMC methods have found several applications in online advertising systems (Agrawal et al., 2013), bio-informatics (Peng et al., 2016; Triguero and Vens, 2016), text classification (Rousu et al., 2006; Mao et al., 2019). The main challenge in HMC is in modelling clas- sification of the document into a large, imbalanced and structured output space. In HMC, the label taxonomy is a partially ordered set (L, ≺) where L is a finite set of all class labels. Relation ≺ refers to is-a relationship between labels, which is asymmet- ric, anti-reflexive and transitive (Silla and Freitas, 2011). Hierarchical structures can provide important insights for learning and …

Content truncated to 500 words