Unsupervised Multi-Domain Adaptation with Feature Embeddings

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 672–682, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Unsupervised Multi-Domain Adaptation with Feature Embeddings Yi Yang and Jacob Eisenstein School of Interactive Computing Georgia Institute of Technology Atlanta, GA 30308 {yiyang+jacobe}@gatech.edu Abstract Representation learning is the dominant tech- nique for unsupervised domain adaptation, but existing approaches have two major weak- nesses. First, they often require the spec- ification of “pivot features” that generalize across domains, which are selected by task- specific heuristics. We show that a novel but simple feature embedding approach provides better performance, by exploiting the feature template structure common in NLP problems. Second, unsupervised domain adaptation is typically treated as a task of moving from a single source to a single target domain. In reality, test data may be diverse, relating to the training data in some ways but not oth- ers. We propose an alternative formulation, in which each instance has a vector of do- main attributes, can be used to learn distill the domain-invariant properties of each feature.1 1 Introduction Domain adaptation is crucial if natural language processing is to be successfully employed in high- impact application areas such as social media, pa- tient medical records, and historical texts. Unsuper- vised domain adaptation is particularly appealing, since it requires no labeled data in the target domain. Some of the most successful approaches to unsu- pervised domain adaptation are based on representa- tion learning: transforming sparse high-dimensional surface features into dense vector representations, 1Source code and a demo are available at https:// github.com/yiyang-gt/feat2vec 1800 1750 1700 1650 1600 1550 1500 Narrative s Letters Dissertation t Theatre Figure 1: Domain graph for the Tycho Brahe cor- pus (Galves and Faria, 2010). Suppose we want to adapt from 19th Century narratives to 16th Century disser- tations: can unlabeled data from other domains help? which are often more robust to domain shift (Blitzer et al., 2006; Glorot et al., 2011). However, these methods are computationally expensive to train, and often require special task-specific heuristics to select good “pivot features.” A second, more subtle challenge for unsupervised domain adaptation is that it is normally framed as adapting from a single source domain to a single tar- get domain. For example, we may be given part- of-speech labeled text from 19th Century narratives, and we hope to adapt the tagger to work on academic dissertations from the 16th Century. This ignores text from the intervening centuries, as well as text that is related by genre, such as 16th Century narra- tives and 19th Century dissertations (see Figure 1). We address a new challenge of unsupervised multi- domain adaptation, where the goal is to leverage this additional unlabeled data to improve performance in the target domain.2 2Multiple domains have been considered in supervised do- main adaptation (e.g., Mansour et al., 2009), but these ap- proaches are not directly applicable when there is no labeled data outside the source domain. 672 …

Content truncated to 500 words