Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 440–447, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification John Blitzer Mark Dredze Department of Computer and Information Science University of Pennsylvania {blitzer|mdredze|pereria@cis.upenn.edu} Fernando Pereira Abstract Automatic sentiment classification has been extensively studied and applied in recent years. However, sentiment is expressed dif- ferently in different domains, and annotating corpora for every possible domain of interest is impractical. We investigate domain adap- tation for sentiment classifiers, focusing on online reviews for different types of prod- ucts. First, we extend to sentiment classifi- cation the recently-proposed structural cor- respondence learning (SCL) algorithm, re- ducing the relative error due to adaptation between domains by an average of 30% over the original SCL algorithm and 46% over a supervised baseline. Second, we identify a measure of domain similarity that corre- lates well with the potential for adaptation of a classifier from one domain to another. This measure could for instance be used to select a small set of domains to annotate whose trained classifiers would transfer well to many other domains. 1 Introduction Sentiment detection and classification has received considerable attention recently (Pang et al., 2002; Turney, 2002; Goldberg and Zhu, 2004). While movie reviews have been the most studied domain, sentiment analysis has extended to a number of new domains, ranging from stock message boards to congressional floor debates (Das and Chen, 2001; Thomas et al., 2006). Research results have been deployed industrially in systems that gauge market reaction and summarize opinion from Web pages, discussion boards, and blogs. With such widely-varying domains, researchers and engineers who build sentiment classification systems need to collect and curate data for each new domain they encounter. Even in the case of market analysis, if automatic sentiment classification were to be used across a wide range of domains, the ef- fort to annotate corpora for each domain may be- come prohibitive, especially since product features change over time. We envision a scenario in which developers annotate corpora for a small number of domains, train classifiers on those corpora, and then apply them to other similar corpora. However, this approach raises two important questions. First, it is well known that trained classifiers lose accuracy when the test data distribution is significantly differ- ent from the training data distribution 1. Second, it is not clear which notion of domain similarity should be used to select domains to annotate that would be good proxies for many other domains. We propose solutions to these two questions and evaluate them on a corpus of reviews for four differ- ent types of products from Amazon: books, DVDs, electronics, and kitchen appliances2. First, we show how to extend the recently proposed structural cor- 1For surveys of recent research on domain adaptation, see the ICML 2006 Workshop on Structural Knowledge Transfer for Machine Learning (http://gameairesearch.uta. edu/) and the NIPS 2006 Workshop on Learning when test and training inputs have …

Content truncated to 500 words