A Non-negative Matrix Factorization Based Approach for Active Dual Supervision from Document and Word Labels

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 949–958, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics A Non-negative Matrix Factorization Based Approach for Active Dual Supervision from Document and Word Labels Chao Shen and Tao Li School of Computing and Information Sciences Florida International University Miami, FL 33199 USA {cshen001,taoli}@cs.fiu.edu Abstract In active dual supervision, not only informa- tive examples but also features are selected for labeling to build a high quality classifier with low cost. However, how to measure the infor- mativeness for both examples and feature on the same scale has not been well solved. In this paper, we propose a non-negative matrix factorization based approach to address this is- sue. We first extend the matrix factorization framework to explicitly model the correspond- ing relationships between feature classes and examples classes. Then by making use of the reconstruction error, we propose a unified scheme to determine which feature or exam- ple a classifier is most likely to benefit from having labeled. Empirical results demonstrate the effectiveness of our proposed methods. 1 Introduction Active learning, as an effective paradigm to optimize the learning benefit from domain experts’ feedback and to reduce the cost of acquiring labeled examples for supervised learning, has been intensively stud- ied in recent years (McCallum and Nigam, 1998; Tong and Koller, 2002; Settles, 2009). Traditional approaches for active learning query the human ex- perts to obtain the labels for intelligently chosen data samples. However, in text classification where the input data is generally represented as document- word matrices, human supervision can be obtained on both documents and words. For example, in sen- timent analysis of product reviews, human labelers can label reviews as positive or negative, they can also label the words that elicit positive sentiment (such as “sensational” and “electrifying”) as posi- tive and words that evoke negative sentiment (such as “depressed” and “unfulfilling”) as negative. Re- cent work has demonstrated that labeled words (or feature supervision) can greatly reduce the number of labeled samples for building high-quality classi- fiers (Druck et al., 2008; Zaidan and Eisner, 2008). In fact, different kinds of supervision generally have different acquisition costs, different degrees of util- ity and are not mutually redundant (Sindhwani et al., 2009). Ideally, effective active learning schemes should be able to utilize different forms of supervi- sion. To incorporate the supervision on words and doc- uments at same time into the active learning scheme, recently an active dual supervision (or dual active learning) has been proposed (Melville and Sind- hwani, 2009; Sindhwani et al., 2009). Comparing with traditional active learning which aims to select the most “informative” examples (e.g., documents) for domain experts to label, active dual supervi- sion selects both the “informative” examples (e.g., documents) and features (e.g., words) for labeling. For active dual supervision to be effective, there are three important components: a) an underlying learning mechanism that is able to learn from both the labeled examples and features (i.e., incorporat- ing …

Content truncated to 500 words