Instance Weighting for Domain Adaptation in NLP

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 264–271, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Instance Weighting for Domain Adaptation in NLP Jing Jiang and ChengXiang Zhai Department of Computer Science University of Illinois at Urbana-Champaign Urbana, IL 61801, USA {jiang4,czhai}@cs.uiuc.edu Abstract Domain adaptation is an important problem in natural language processing (NLP) due to the lack of labeled data in novel domains. In this paper, we study the domain adaptation problem from the instance weighting per- spective. We formally analyze and charac- terize the domain adaptation problem from a distributional view, and show that there are two distinct needs for adaptation, cor- responding to the different distributions of instances and classification functions in the source and the target domains. We then propose a general instance weighting frame- work for domain adaptation. Our empir- ical results on three NLP tasks show that incorporating and exploiting more informa- tion from the target domain through instance weighting is effective. 1 Introduction Many natural language processing (NLP) problems such as part-of-speech (POS) tagging, named entity (NE) recognition, relation extraction, and seman- tic role labeling, are currently solved by supervised learning from manually labeled data. A bottleneck problem with this supervised learning approach is the lack of annotated data. As a special case, we often face the situation where we have a sufficient amount of labeled data in one domain, but have little or no labeled data in another related domain which we are interested in. We thus face the domain adap- tation problem. Following (Blitzer et al., 2006), we call the first the source domain, and the second the target domain. The domain adaptation problem is commonly en- countered in NLP. For example, in POS tagging, the source domain may be tagged WSJ articles, and the target domain may be scientific literature that con- tains scientific terminology. In NE recognition, the source domain may be annotated news articles, and the target domain may be personal blogs. Another example is personalized spam filtering, where we may have many labeled spam and ham emails from publicly available sources, but we need to adapt the learned spam filter to an individual user’s inbox be- cause the user has her own, and presumably very dif- ferent, distribution of emails and notion of spams. Despite the importance of domain adaptation in NLP, currently there are no standard methods for solving this problem. An immediate possible solu- tion is semi-supervised learning, where we simply treat the target instances as unlabeled data but do not distinguish the two domains. However, given that the source data and the target data are from dif- ferent distributions, we should expect to do better by exploiting the domain difference. Recently there have been some studies addressing domain adapta- tion from different perspectives (Roark and Bacchi- ani, 2003; Chelba and Acero, 2004; Florian et al., 2004; Daum´e III and Marcu, 2006; Blitzer et al., 2006). However, there have not been many studies that focus on the …

Content truncated to 500 words