HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3259–3265 June 6–11, 2021. ©2021 Association for Computational Linguistics 3259 HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization Zhongfen Deng1, Hao Peng2, 3, Dongxiao He4, Jianxin Li2, Philip S. Yu1 1Department of Computer Science, University of Illinois at Chicago, Chicago, USA 2BDBC, Beihang University, Beijing, China 3School of Cyber Science and Technology, Beihang University, Beijing, China 4School of Computer Science and Technology, Tianjin University, Tianjin, China {zdeng21,psyu}@uic.edu, {penghao,lijx}@act.buaa.edu.cn hedongxiao@tju.edu.cn Abstract The current state-of-the-art model HiAGM for hierarchical text classification has two limi- tations. First, it correlates each text sample with all labels in the dataset which contains irrelevant information. Second, it does not consider any statistical constraint on the la- bel representations learned by the structure encoder, while constraints for representation learning are proved to be helpful in previous work. In this paper, we propose HTCInfoMax to address these issues by introducing infor- mation maximization which includes two mod- ules: text-label mutual information maximiza- tion and label prior matching. The first mod- ule can model the interaction between each text sample and its ground truth labels explic- itly which filters out irrelevant information. The second one encourages the structure en- coder to learn better representations with de- sired characteristics for all labels which can better handle label imbalance in hierarchical text classification. Experimental results on two benchmark datasets demonstrate the effec- tiveness of the proposed HTCInfoMax. 1 Introduction Hierarchical text classification (HTC) is a partic- ular subtask of multi-label text classification (Li et al., 2020). Many datasets have been proposed to study HTC for decades, such as RCV1 (Lewis et al., 2004) and NYTimes (Sandhaus, 2008), which cate- gorize a news into several categories/labels. And all the labels in each dataset are usually organized as a tree or a directed acyclic graph. Thus, there is a label taxonomic hierarchy existing in each dataset. The goal of HTC is to predict multiple labels in a given label hierarchy for a given text. There are two groups of existing methods for HTC: local approaches and global approaches. Lo- cal approaches usually build a classifier for each label/node (Banerjee et al., 2019), or for each par- ent node, or for each level of the label hierar- chy(Wehrmann et al., 2018; Huang et al., 2019; Chang et al., 2020). Global approaches just build one classifier to simultaneously predict multiple la- bels of a given text. The earlier global approaches ignore the hierarchical structure of labels and as- sume there is no dependency among labels which leads to flat models such as (Johnson and Zhang, 2015). Later on, more and more works try to make use of the label taxonomic hierarchy to improve the performance by employing different strategies such as recursively regularized Graph-CNN (Peng et al., 2018), reinforcement learning (Mao et al., 2019), attentional capsule network (Peng et al., 2019), meta-learning (Wu et al., 2019) and structure en- coder (Zhou et …

Content truncated to 500 words