HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7809–7821 July 9-14, 2023 ©2023 Association for Computational Linguistics HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification He Zhu1∗, Chong Zhang1∗, Junjie Huang1, Junran Wu1†, Ke Xu1,2 1State Key Lab of Software Development Environment Beihang University, Beijing, 100191, China 2Zhongguancun Laboratory, Beijing, 100094, China {roy_zh, chongzh, huangjunjie, wu_junran, kexu}@buaa.edu.cn Abstract Hierarchical text classification (HTC) is a chal- lenging subtask of multi-label classification as the labels form a complex hierarchical structure. Existing dual-encoder methods in HTC achieve weak performance gains with huge memory overheads and their structure encoders heav- ily rely on domain knowledge. Under such observation, we tend to investigate the feasi- bility of a memory-friendly model with strong generalization capability that could boost the performance of HTC without prior statistics or label semantics. In this paper, we propose Hierarchy-aware Tree Isomorphism Network (HiTIN) to enhance the text representations with only syntactic information of the label hi- erarchy. Specifically, we convert the label hier- archy into an unweighted tree structure, termed coding tree, with the guidance of structural en- tropy. Then we design a structure encoder to incorporate hierarchy-aware information in the coding tree into text representations. Besides the text encoder, HiTIN only contains a few multi-layer perceptions and linear transforma- tions, which greatly saves memory. We conduct experiments on three commonly used datasets and the results demonstrate that HiTIN could achieve better test performance and less mem- ory consumption than state-of-the-art (SOTA) methods. 1 Introduction Hierarchical text classification is a sub-task of text multi-label classification, which is commonly ap- plied in scenarios such as news document clas- sification (Lewis et al., 2004; Sandhaus, Evan, 2008), academic paper classification (Kowsari et al., 2017), and so on. Unlike traditional classifi- cation tasks, the labels of HTC have parent-child relationships forming a hierarchical structure. Due to the complex structure of label hierarchy and the ∗Equal Contribution. †Correspondence to: Junran Wu. �� �� �� �� �� �� �("��%�$���%� #��!����%�"�'�%&���� ���� ���� ���� ���� ���� ���� ���� � �%$����&�$%� � ������(%&� ��)'���� � ������ � ������ � ��'�� ����#�$��) Figure 1: Micro-F1 score and the number of trainable parameters of our method and SOTAs with dual en- coders on Web Of Science dataset. imbalanced frequency of labels, HTC becomes a challenging task in natural language processing. Recent studies in HTC typically utilize a dual- encoder framework (Zhou et al., 2020), which con- sists of a text encoder for text representations and a structure encoder to inject the information of labels into text. The text encoder could be a traditional backbone for text classification, for instance, Tex- tRCNN (Lai et al., 2015) or BERT (Devlin et al., 2019). The structure encoder is a Graph Neural Network (GNN) that treats the label hierarchy as a Directed Acyclic Graph (DAG) and propagates the information among labels. To maximize the propa- gation ability of the structure encoder, Zhou et al. (2020) learn textual features of labels and count the prior …

Content truncated to 500 words