Frustratingly Easy Domain Adaptation

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 256–263, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Frustratingly Easy Domain Adaptation Hal Daum´e III School of Computing University of Utah Salt Lake City, Utah 84112 me@hal3.name Abstract We describe an approach to domain adapta- tion that is appropriate exactly in the case when one has enough “target” data to do slightly better than just using only “source” data. Our approach is incredibly simple, easy to implement as a preprocessing step (10 lines of Perl!) and outperforms state- of-the-art approaches on a range of datasets. Moreover, it is trivially extended to a multi- domain adaptation problem, where one has data from a variety of different domains. 1 Introduction The task of domain adaptation is to develop learn- ing algorithms that can be easily ported from one domain to another—say, from newswire to biomed- ical documents. This problem is particularly inter- esting in NLP because we are often in the situation that we have a large collection of labeled data in one “source” domain (say, newswire) but truly desire a model that performs well in a second “target” do- main. The approach we present in this paper is based on the idea of transforming the domain adaptation learning problem into a standard supervised learn- ing problem to which any standard algorithm may be applied (eg., maxent, SVMs, etc.). Our transfor- mation is incredibly simple: we augment the feature space of both the source and target data and use the result as input to a standard learning algorithm. There are roughly two varieties of the domain adaptation problem that have been addressed in the literature: the fully supervised case and the semi- supervised case. The fully supervised case mod- els the following scenario. We have access to a large, annotated corpus of data from a source do- main. In addition, we spend a little money to anno- tate a small corpus in the target domain. We want to leverage both annotated datasets to obtain a model that performs well on the target domain. The semi- supervised case is similar, but instead of having a small annotated target corpus, we have a large but unannotated target corpus. In this paper, we focus exclusively on the fully supervised case. One particularly nice property of our approach is that it is incredibly easy to implement: the Ap- pendix provides a 10 line, 194 character Perl script for performing the complete transformation (avail- able at http://hal3.name/easyadapt.pl.gz). In addition to this simplicity, our algorithm performs as well as (or, in some cases, better than) current state of the art techniques. 2 Problem Formalization and Prior Work To facilitate discussion, we first introduce some no- tation. Denote by X the input space (typically either a real vector or a binary vector), and by Y the output space. We will write Ds to denote the distribution over source examples and Dt to denote the distri- bution over target examples. We assume access to …

Content truncated to 500 words