Multi-label Text Categorization with Joint Learning Predictions-as-Features Method

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 835–839, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Multi-label Text Categorization with Joint Learning Predictions-as-Features Method Li Li1 Baobao Chang1 Shi Zhao2 Lei Sha1 Xu Sun1 Houfeng Wang1 Key Laboratory of Computational Linguistics(Peking University), Ministry of Education, China1 Key Laboratory on Machine Perception(Peking University), Ministry of Education, China2 {li.l, chbb, shalei, z.s, xusun, wanghf}@pku.edu.cn Abstract Multi-label text categorization is a type of text categorization, where each document is assigned to one or more categories. Re- cently, a series of methods have been de- veloped, which train a classifier for each label, organize the classifiers in a partially ordered structure and take predictions pro- duced by the former classifiers as the latter classifiers’ features. These predictions-as- features style methods model high order label dependencies and obtain high per- formance. Nevertheless, the predictions- as-features methods suffer a drawback. When training a classifier for one label, the predictions-as-features methods can mod- el dependencies between former labels and the current label, but they can’t model de- pendencies between the current label and the latter labels. To address this problem, we propose a novel joint learning algorith- m that allows the feedbacks to be propa- gated from the classifiers for latter labels to the classifier for the current label. We conduct experiments using real-world tex- tual data sets, and these experiments il- lustrate the predictions-as-features models trained by our algorithm outperform the o- riginal models. 1 Introduction The multi-label text categorization is a type of text categorization, where each document is assigned to one or more categories simultaneously. The multi-label setting is common and useful in the re- al world. For example, in the news categorization task, a newspaper article concerning global warm- ing can be classified into two categories simul- taneously, namely environment and science. For another example, in the task of classifying mu- sic lyrics into emotions, a song’s lyrics can deliv- er happiness and excitement simultaneously. The research about the multi-label text categorization attracts increasing attention (Srivastava and Zane- Ulman, 2005; Katakis et al., 2008; Rubin et al., 2012; Nam et al., 2013; Li et al., 2014). Recently, a series of predictions-as-features style methods have been developed, which train a classifier for each label, organize the classifiers in a partially ordered structure and take prediction- s produced by the former classifiers as the latter classifiers’ features. These predictions-as-features style methods model high order label dependen- cies (Zhang and Zhang, 2010) and obtain high performance. Classifier chain (CC) (Read et al., 2011) and multi-label Learning by Exploiting lA- bel Dependency (Lead) (Zhang and Zhang, 2010) s.are two famous predictions-as-features method- CC organizes classifiers along a chain and LEAD organizes classifiers in a Bayesian net- work. Besides, there are other works on extend- ing the predictions-as-features methods (Zaragoza et al., 2011; Gonc¸alves et al., 2013; Sucar et al., 2014). In this paper, we focus on the predictions- as-features style methods. The previous works of the predictions-as- features methods …

Content truncated to 500 words