Exploring Label Hierarchy in a Generative Way for Hierarchical Text Classification

Proceedings of the 29th International Conference on Computational Linguistics, pages 1116–1127 October 12–17, 2022. 1116 Exploring Label Hierarchy in a Generative Way for Hierarchical Text Classification Wei Huang1∗, Chen Liu2†, Bo Xiao2, Yihua Zhao2, Zhaoming Pan2, Zhimin Zhang2, Xinyun Yang2, Guiquan Liu1 1University of Science and Technology of China 2NetEase Media Technology (Beijing) Co., Ltd. hw001@mail.ustc.edu.cn, gqliu@ustc.edu.cn {liuchen5,xiaobo02,hzzhaoyihua}@corp.netease.com {panzhaoming,zhangzhimin,yangxinyun}@corp.netease.com Abstract Hierarchical Text Classification (HTC), which aims to predict text labels organized in hier- archical space, is a significant task lacking in investigation in natural language processing. Existing methods usually encode the entire hier- archical structure and fail to construct a robust label-dependent model, making it hard to make accurate predictions on sparse lower-level la- bels and achieving low Macro-F1. In this paper, we explore the level dependency and path de- pendency of the label hierarchy in a generative way for building the knowledge of upper-level labels of current path into lower-level ones, and thus propose a novel PAAM-HiA-T5 model for HTC: a hierarchy-aware T5 model with path- adaptive attention mechanism. Specifically, we generate a multi-level sequential label structure to exploit hierarchical dependency across differ- ent levels with Breadth-First Search (BFS) and T5 model. To further improve label dependency prediction within each path, we then propose an original path-adaptive attention mechanism (PAAM) to lead the model to adaptively focus on the path where the currently generated la- bel is located, shielding the noise from other paths. Comprehensive experiments on three benchmark datasets show that PAAM-HiA-T5 greatly outperforms all state-of-the-art HTC approaches especially in Macro-F1. 1 Introduction Hierarchical text classification (HTC), where text labels are predicted within a hierarchical structure, is a challenging task that has not yet received due attention within the field of multi-label classifica- tion. HTC methods have been extensively applied in industry domains, e.g., news article classifica- tion (Sandhaus, 2008), product classification in E- commerce (Yu et al., 2018), bidding strategy in paid search marketing (Agrawal et al., 2013). *Contribution during internship at NetEase Inc. †Corresponding Author Root News Features Arts Dance Music Opinions Sports Business Hockey Baseball Football Root News Features Arts Dance Music Opinions Sports Business Hockey Baseball Football Root News Features Arts Dance Music Opinions Sports Business Hockey Baseball Football (a) Overall hierarchy (b) Level dependency (c) Path dependency Figure 1: (a): the static labeling process is uniform and simultaneous for all labels in the label hierarchy. (b): the dynamic labeling process where the lower-level la- bels depend on the upper-level labels. (c): the dynamic labeling process focuses on ancestor labels already gen- erated on the current path. In HTC tasks, labels at lower-level are inevitably sparse due to the hierarchical structure. Many stud- ies (Barbedo and Lopes, 2006; Xiao et al., 2007; Johnson and Zhang, 2015) completely or partially neglect such hierarchical structure and fail to ac- curately predict those lower-level labels, achieving low Macro-F1 score. Existing studies (Peng et al., 2021; Wu et al., 2019) have proved that introduc- ing structure information can boost the predictive power on low-level labels and thus improve …

Content truncated to 500 words