Domain Adaptation by Constraining Inter-Domain Variability of Latent Feature Representation

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 62–71, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Domain Adaptation by Constraining Inter-Domain Variability of Latent Feature Representation Ivan Titov Saarland University Saarbruecken, Germany titov@mmci.uni-saarland.de Abstract We consider a semi-supervised setting for do- main adaptation where only unlabeled data is available for the target domain. One way to tackle this problem is to train a generative model with latent variables on the mixture of data from the source and target domains. Such a model would cluster features in both do- mains and ensure that at least some of the la- tent variables are predictive of the label on the source domain. The danger is that these pre- dictive clusters will consist of features specific to the source domain only and, consequently, a classifier relying on such clusters would per- form badly on the target domain. We in- troduce a constraint enforcing that marginal distributions of each cluster (i.e., each latent variable) do not vary significantly across do- mains. We show that this constraint is effec- tive on the sentiment classification task (Pang et al., 2002), resulting in scores similar to the ones obtained by the structural correspon- dence methods (Blitzer et al., 2007) without the need to engineer auxiliary tasks. 1 Introduction Supervised learning methods have become a stan- dard tool in natural language processing, and large training sets have been annotated for a wide vari- ety of tasks. However, most learning algorithms op- erate under assumption that the learning data orig- inates from the same distribution as the test data, though in practice this assumption is often violated. This difference in the data distributions normally re- sults in a significant drop in accuracy. To address this problem a number of domain-adaptation meth- ods has recently been proposed (see e.g., (Daum´e and Marcu, 2006; Blitzer et al., 2006; Bickel et al., 2007)). In addition to the labeled data from the source domain, they also exploit small amounts of labeled data and/or unlabeled data from the target domain to estimate a more predictive model for the target domain. In this paper we focus on a more challenging and arguably more realistic version of the domain- adaptation problem where only unlabeled data is available for the target domain. One of the most promising research directions on domain adaptation for this setting is based on the idea of inducing a shared feature representation (Blitzer et al., 2006), that is mapping from the initial feature representa- tion to a new representation such that (1) examples from both domains ‘look similar’ and (2) an accu- rate classifier can be trained in this new representa- tion. Blitzer et al. (2006) use auxiliary tasks based on unlabeled data for both domains (called pivot fea- tures) and a dimensionality reduction technique to induce such shared representation. The success of their domain-adaptation method (Structural Corre- spondence Learning, SCL) crucially depends on the choice of the auxiliary tasks, and defining them can be a …

Content truncated to 500 words