Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1190–1200 August 1–6, 2021. ©2021 Association for Computational Linguistics 1190 Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning Ximing Zhang1∗ †, Qian-Wen Zhang2∗, Zhao Yan2, Ruifang Liu1, Yunbo Cao2 1Beijing University of Posts and Telecommunications, Beijing 100876, China 2Tencent Cloud Xiaowei, Beijing 100080, China ximingzhang@bupt.edu.cn, cowenzhang@tencent.com, zhaoyan@tencent.com, lrf@bupt.edu.cn, yunbocao@tencent.com Abstract In multi-label text classification (MLTC), each given document is associated with a set of correlated labels. To capture label correla- tions, previous classifier-chain and sequence- to-sequence models transform MLTC to a se- quence prediction task. However, they tend to suffer from label order dependency, la- bel combination over-fitting and error prop- agation problems. To address these prob- lems, we introduce a novel approach with multi-task learning to enhance label correla- tion feedback. We first utilize a joint em- bedding (JE) mechanism to obtain the text and label representation simultaneously. In MLTC task, a document-label cross atten- tion (CA) mechanism is adopted to gener- ate a more discriminative document represen- tation. Furthermore, we propose two auxil- iary label co-occurrence prediction tasks to en- hance label correlation learning: 1) Pairwise Label Co-occurrence Prediction (PLCP), and 2) Conditional Label Co-occurrence Predic- tion (CLCP). Experimental results on AAPD and RCV1-V2 datasets show that our method outperforms competitive baselines by a large margin. We analyze low-frequency label per- formance, label dependency, label combina- tion diversity and coverage speed to show the effectiveness of our proposed method on label correlation learning. Our code is available at https://github.com/EiraZhang/LACO. 1 Introduction Multi-label text classification (MLTC) is an impor- tant natural language processing task with applica- tions in text categorization, information retrieval, web mining, and many other real-world scenar- ios (Zhang and Zhou, 2014; Liu et al., 2020). In MLTC, each given document is associated with a ∗Equal contribution. †Work done during an internship at Tencent. set of labels which are often related statistically and semantically. Label correlations should be sufficiently utilized to build multi-label classifi- cation models with strong generalization perfor- mance (Tsoumakas et al., 2009; Gibaja and Ven- tura, 2015). In particular, learning the dependen- cies between labels might be helpful in modeling the low-frequency labels, because real-world clas- sification problems tend to exhibit long-tail label distribution, where low-frequency labels are asso- ciated with only a few instances and are difficult to learn (Menon et al., 2020). Previous sequence-to-sequence (Seq2Seq) based methods (Nam et al., 2017; Yang et al., 2018) have been shown to have a powerful ability to capture la- bel correlations with using the current hidden state of the model and the prefix label predictions. How- ever, exposure bias phenomenon (Bengio et al., 2015) may cause the models overfit to the frequent label sequence in training set, thus lead to several problems. First, Seq2Seq-based methods heavily rely on a predefined ordering of labels and perform sensitively to the label order (Vinyals et al.; Yang et al., 2019; Qin et al., 2019). Actually, labels are essentially an order-independent set in the MLTC task. …

Content truncated to 500 words