Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2935–2943, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2935 Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs Jueqing Lu1, Lan Du1∗, Ming Liu2, and Joanna Dipnall3 1Faculty of Information Technology, Monash University, Australia, VIC 3800 2School of Information Technology, Deakin University, Australia, VIC 3217 3School of Public Health and Preventive Medicine, Monash University, Australia, VIC 3800 jluu0014@student.monash.edu {lan.du, jo.dipnall}@monash.edu m.liu@deakin.edu.au Abstract Few/Zero-shot learning is a big challenge of many classifications tasks, where a classifier is required to recognise instances of classes that have very few or even no training sam- ples. It becomes more difficult in multi- label classification, where each instance is la- belled with more than one class. In this pa- per, we present a simple multi-graph aggrega- tion model that fuses knowledge from multi- ple label graphs encoding different semantic label relationships in order to study how the aggregated knowledge can benefit multi-label zero/few-shot document classification. The model utilises three kinds of semantic informa- tion, i.e., the pre-trained word embeddings, la- bel description, and pre-defined label relations. Experimental results derived on two large clin- ical datasets (i.e., MIMIC-II and MIMIC-III ) and the EU legislation dataset show that meth- ods equipped with the multi-graph knowledge aggregation achieve significant performance improvement across almost all the measures on few/zero-shot labels. 1 Introduction Multi-label learning is a fundamental and practical problem in computer vision and natural language processing. Many tasks, such as automated medical coding (Yan et al., 2010; Rios and Kavuluru, 2018; Du et al., 2019), recommender systems (Halder et al., 2018), image classification (Chen et al., 2019; Wang et al., 2020), law study (Parikh et al., 2019; Chalkidis et al., 2019), and stance detection (Fer- reira and Vlachos, 2019) can be formulated as a multi-label learning problem. Different from multi- class classification, an instance in multi-label learn- ing is often associated with more than one class label, which makes the task even more challenging due to the combinatorial nature of the label space. ∗Corresponding author i.e., the number of possible label combinations is exponential with the total number of labels. In real-world applications, there are often insuf- ficient or even unavailable training data of ever emerging classes (Vinyals et al., 2016; Xian et al., 2019). For instance, more than half of the Interna- tional Classification of Diseases (ICD) codes are not associated with a discharge summary in the MIMIC-III dataset (Johnson et al., 2016; Rios and Kavuluru, 2018). As a solution, zero-shot learning (Xian et al., 2019; Wang et al., 2019) aims to gen- eralize classifiers to unseen classes by leveraging various label semantics. Those classifiers are re- quired to recognise instances of classes that have never been seen in the training set, which becomes more difficult in multi-label learning. Moreover, the number of classes can reach hun- dreds of thousands. The ICD-9-CM taxonomy contains 17K diagnosis/procedure codes1, where the majority occurs less than 10 times in MIMIC- III; the EU legislation …

Content truncated to 500 words