Active Learning for Imbalanced Sentiment Classification

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 139–148, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Active Learning for Imbalanced Sentiment Classification Shoushan Li†, Shengfeng Ju†, Guodong Zhou† Xiaojun Li‡ †Natural Language Processing Lab School of Computer Science and Technology ‡College of Computer and Information Engineering Soochow University, Suzhou, 215006, China Zhejiang Gongshang University {shoushan.li, shengfeng.ju}@gmail.com, Hangzhou, 310035, China gdzhou@suda.edu.cn lixj@mail.zjgsu.edu.cn Abstract Active learning is a promising way for sentiment classification to reduce the annotation cost. In this paper, we focus on the imbalanced class distribution scenario for sentiment classification, wherein the number of positive samples is quite different from that of negative samples. This scenario posits new challenges to active learning. To address these challenges, we propose a novel active learning approach, named co-selecting, by taking both the imbalanced class distribution issue and uncertainty into account. Specifically, our co-selecting approach employs two feature subspace classifiers to collectively select most informative minority-class samples for manual annotation by leveraging certainty measurement and an uncertainty a measurement, and in the meanwhile, automatically label most informative majority-class samples, to reduce human- annotation efforts. Extensive experiments across four domains demonstrate great potential and effectiveness of our proposed co-selecting approach to active learning for imbalanced sentiment classification. 1 1 Introduction Sentiment classification is the task of identifying the sentiment polarity (e.g., positive or negative) of *1 Corresponding author a natural language text towards a given topic (Pang et al., 2002; Turney, 2002) and has become the core component of many important applications in opinion analysis (Cui et al., 2006; Li et al., 2009; Lloret et al., 2009; Zhang and Ye, 2008). Most of previous studies in sentiment classification focus on learning models from a large number of labeled data. However, in many real-world applications, manual annotation is expensive and time-consuming. In these situations, active learning approaches could be helpful by actively selecting most informative samples for manual annotation. Compared to traditional active learning for sentiment classification, active learning for imbalanced sentiment classification faces some unique challenges. As a specific type of sentiment classification, imbalanced sentiment classification deals with the situation in which there are many more samples of one class (called majority class) than the other class (called minority class), and has attracted much attention due to its high realistic value in real-world applications (Li et al., 2011a). In imbalanced sentiment classification, since the minority-class samples (denoted as MI samples) are normally much sparse and thus more precious and informative for learning compared to the majority-class ones (denoted as MA samples), it is worthwhile to spend more on manually annotating MI samples to guarantee both the quality and quantity of MI samples. Traditionally, uncertainty has been popularly used as a basic measurement in active learning (Lewis and Gale, 2004). Therefore, how to select most informative MI samples for manual annotation without violating the basic 139 uncertainty requirement in active learning is challenging in imbalanced sentiment classification. In this paper, we …

Content truncated to 500 words