A POS-based Ensemble Model for Cross-domain Sentiment Classification

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 614–622, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP A POS-based Ensemble Model for Cross-domain Sentiment Classification Rui Xia and Chengqing Zong National Laboratory of Pattern Recognition Institute of Automation, Chinese Academy of Sciences rxia.cn@gmail.com, cqzong@nlpr.ia.ac.cn Abstract In this paper, we focus on the tasks of cross-domain sentiment classification. We find across different domains, features with some types of part-of-speech (POS) tags are domain-dependent, while some others are domain-free. Based on this finding, we proposed a POS-based ensemble model to efficiently integrate features with different types of POS tags to improve the classifica- tion performance. Weights are trained by stochastic gradient descent (SGD) to opti- mize the perceptron and minimal classifica- tion error (MCE) criteria. Experimental results show that the proposed ensemble model is quite effective for the task of cross-domain sentiment classification. 1 Introduction In recent years, transfer learning and domain adap- tation, the task aiming to utilize labeled data from the other domains (source domain) to help learning for current domain (target domain), has attracted more and more attention in the fields of both ma- chine learning and natural language processing, including sentiment classification. The task of sen- timent classification is supposed to be domain- specific. Classifiers trained on the source domain usually perform poorly in the target domain. This is quite reasonable since the word distribution changes from one domain to another, and some words that are positive in one domain may express an opposite meaning in another one. Therefore, it is challenging to transfer a classifier trained on the source domain to the target domain. Methodology to solve this problem can be di- vided into three major categories (Pan and Yang, 2009): the instance-based transfer, the feature- based transfer and the parameter-based transfer. The instance-based transfer learns the importance of labeled data in the source domain by instance re-weighting and importance sampling. These re- weighed instances are then used for learning in the target domain. Feature-based transfer aims to learn a good feature representation for the target domain using labeled data in the source domain with the help of a large number of unlabeled data in the tar- get domain. The parameter-based transfer mostly assumes that individual models for related tasks share some parameters or prior distribution of hy- per-parameters. The shared part is then added to the cost function for transfer learning. In this paper, we propose a POS-based ensemble model for cross-domain sentiment classification. Other than the above-mentioned methodology, the transfer procedure in our approach is neither in- stance re-weighting nor feature representation. Broadly speaking, our approach belongs to the pa- rameter-based transfer, but different from the tradi- tional ways, the transfer procedure is embodied in an ensemble manner. By observing the K-L distance of multi-domain datasets, we find that cross different domains, the distribution of features with some types of POS tags, such as adjectives and adverbs, has little 614 change; while some other parts, for example, nouns, vary …

Content truncated to 500 words