Muli-label Text Categorization with Hidden Components

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1816–1821, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Muli-label Text Categorization with Hidden Components Li Li Longkai Zhang Houfeng Wang Key Laboratory of Computational Linguistics (Peking University) Ministry of Education, China li.l@pku.edu.cn, zhlongk@qq.com, wanghf@pku.edu.cn Abstract Multi-label text categorization (MTC) is supervised learning, where a documen- t may be assigned with multiple categories (labels) simultaneously. The labels in the MTC are correlated and the correlation re- sults in some hidden components, which represent the ”share” variance of correlat- ed labels. In this paper, we propose a method with hidden components for MTC. The proposed method employs PCA to capture the hidden components, and incor- porates them into a joint learning frame- work to improve the performance. Experi- ments with real-world data sets and evalu- ation metrics validate the effectiveness of the proposed method. 1 Introduction Many real-world text categorization applications are multi-label text categorization (Srivastava and Zane-Ulman, 2005; Katakis et al., 2008; Rubin et al., 2012; Nam et al., 2013), where a docu- ments is usually assigned with multiple labels si- multaneously. For example, as figure 1 shows, a newspaper article concerning global warming can be classified into two categories, Environmen- t, and Science simultaneously. Let X = Rd be the documents corpus, and Y = {0, 1}m be the label space with m labels. We denote by {(x1 x1 x1,y1 y1 y1), (x2 x2 x2,y2 y2 y2), ..., (xn xn xn,yn yn yn)} the training set of n documents. Each document is denoted by a vec- tor xxxi = [xi,1, xi,2, ..., xi,d] of d dimensions. The labeling of the i-th document is denoted by vector yyyi = [yi,1, yi,2, ..., yi,m], where yil is 1 when the i-th document has the l-th label and 0 otherwise. The goal is to learn a function fff : X → Y. Gener- ally, we can assume fff consists of m functions, one for a label. fff = [f1, f2, ..., fm] Figure 1: A newspaper article concerning global warming can be classified into two categories, En- vironment, and Science. The labels in the MLC are correlated. For ex- ample, a ”politics” document is likely to be an ”e- conomic” document simultaneously, but likely not to be a ”literature” document. According to the latent variable model (Tabachnick et al., 2001), the labels with correlation result in some hidden components, which represent the ”share” variance of correlated labels. Intuitively, if we can capture and utilize these hidden components in MTC, the performance will be improved. To implement this idea, we propose a multi- label text categorization method with hidden components, which employ PCA to capture the hidden components, and then incorporates these hidden components into a joint learning framework. Experiments with various da- ta sets and evaluation metrics validate the values of our method. The research close to our work is ML-LOC (Multi-Label learning using LOcal Cor- relation) in (Huang and Zhou, 2012). The differ- 1816 ences …

Content truncated to 500 words