Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4354–4364, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4354 Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification Jiawei Wu and Wenhan Xiong and William Yang Wang Department of Computer Science University of California, Santa Barbara Santa Barbara, CA 93106 USA {jiawei wu,xwhan,william}@cs.ucsb.edu Abstract Many tasks in natural language processing can be viewed as multi-label classification prob- lems. However, most of the existing mod- els are trained with the standard cross-entropy loss function and use a fixed prediction pol- icy (e.g., a threshold of 0.5) for all the la- bels, which completely ignores the complexity and dependencies among different labels. In this paper, we propose a meta-learning method to capture these complex label dependencies. More specifically, our method utilizes a meta- learner to jointly learn the training policies and prediction policies for different labels. The training policies are then used to train the classifier with the cross-entropy loss function, and the prediction policies are further imple- mented for prediction. Experimental results on fine-grained entity typing and text classi- fication demonstrate that our proposed method can obtain more accurate multi-label classifi- cation results. 1 Introduction Multi-label classification aims at learning to make predictions on instances that are associated with multiple labels simultaneously, whereas in a clas- sic multi-class classification setting, typically one instance has only one label. Multi-label classifi- cation is a common learning paradigm in a large amount of real-world natural language process- ing (NLP) applications, such as fine-grained en- tity typing (Ling and Weld, 2012; Shimaoka et al., 2017; Abhishek et al., 2017; Xin et al., 2018) and text classification (Nam et al., 2014; Liu et al., 2017; Chen et al., 2017; Wu et al., 2018). Significant amounts of research studies have been dedicated to tackle the multi-label classifica- tion problem (Zhang and Zhou, 2014), from tradi- tional statistical models (Zhang and Zhou, 2007; Zhou et al., 2012; Surdeanu et al., 2012) to neu- (1) (2) /organization /company /government /news /broadcast /person /artist /author /political_figure /buildings /economics /World Expo /business /urban Figure 1: Illustration of complex label dependencies in multi-label classification. (1) Labels are aligned with some knowledge graphs. In this case, labels are orga- nized in a hierarchical dependency structure. (2) Even if labels do not demonstrate explicit correlation, they can still have implicit semantic dependencies. ral network-based models (Nam et al., 2014; Shi- maoka et al., 2017). These models have variable structures, but usually, share the standard cross- entropy loss function for training. After training, these models tend to use a single prediction pol- icy for all the labels to generate the final predic- tions. Actually, the above process is based on the assumption that there is no dependency among the labels. However, as shown in Figure 1, this assumption is hard to be satisfied in real-world datasets, and the complex label dependencies re- ceive little attention in …

Content truncated to 500 words