When Specialists and Generalists Work Together: Overcoming Domain Dependence in Sentiment Tagging

Proceedings of ACL-08: HLT, pages 290–298, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics When Specialists and Generalists Work Together: Overcoming Domain Dependence in Sentiment Tagging Alina Andreevskaia Concordia University Montreal, Quebec andreev@cs.concordia.ca Sabine Bergler Concordia University Montreal, Canada bergler@cs.concordia.ca Abstract This study presents a novel approach to the problem of system portability across differ- ent domains: a sentiment annotation system that integrates a corpus-based classifier trained on a small set of annotated in-domain data and a lexicon-based system trained on Word- Net. The paper explores the challenges of sys- tem portability across domains and text gen- res (movie reviews, news, blogs, and product reviews), highlights the factors affecting sys- tem performance on out-of-domain and small- set in-domain data, and presents a new sys- tem consisting of the ensemble of two classi- fiers with precision-based vote weighting, that provides significant gains in accuracy and re- call over the corpus-based classifier and the lexicon-based system taken individually. 1 Introduction One of the emerging directions in NLP is the de- velopment of machine learning methods that per- form well not only on the domain on which they were trained, but also on other domains, for which training data is not available or is not sufficient to ensure adequate machine learning. Many applica- tions require reliable processing of heterogeneous corpora, such as the World Wide Web, where the diversity of genres and domains present in the Inter- net limits the feasibility of in-domain training. In this paper, sentiment annotation is defined as the assignment of positive, negative or neutral senti- ment values to texts, sentences, and other linguistic units. Recent experiments assessing system porta- bility across different domains, conducted by Aue and Gamon (2005), demonstrated that sentiment an- notation classifiers trained in one domain do not per- form well on other domains. A number of methods has been proposed in order to overcome this system portability limitation by using out-of-domain data, unlabelled in-domain corpora or a combination of in-domain and out-of-domain examples (Aue and Gamon, 2005; Bai et al., 2005; Drezde et al., 2007; Tan et al., 2007). In this paper, we present a novel approach to the problem of system portability across different do- mains by developing a sentiment annotation sys- tem that integrates a corpus-based classifier with a lexicon-based system trained on WordNet. By adopting this approach, we sought to develop a system that relies on both general and domain- specific knowledge, as humans do when analyzing a text. The information contained in lexicographi- cal sources, such as WordNet, reflects a lay person’s general knowledge about the world, while domain- specific knowledge can be acquired through classi- fier training on a small set of in-domain data. The first part of this paper reviews the extant lit- erature on domain adaptation in sentiment analy- sis and highlights promising directions for research. The second part establishes a baseline for system evaluation by drawing comparisons of system per- formance across four different domains/genres - movie reviews, news, blogs, and product reviews. The …

Content truncated to 500 words