Self-Adaptation for Unsupervised Domain Adaptation

Proceedings of Recent Advances in Natural Language Processing, pages 213–222, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_025 213 Self-Adaptation for Unsupervised Domain Adaptation Xia Cui and Danushka Bollegala Department of Computer Science University of Liverpool Ashton Street, Liverpool L69 3BX, United Kingdom {xia.cui, danushka.bollegala}@liverpool.ac.uk Abstract Lack of labelled data in the target do- main for training is a common problem in domain adaptation. To overcome this problem, we propose a novel unsuper- vised domain adaptation method that com- bines projection and self-training based approaches. Using the labelled data from the source domain, we first learn a pro- jection that maximises the distance among the nearest neighbours with opposite la- bels in the source domain. Next, we project the source domain labelled data us- ing the learnt projection and train a clas- sifier for the target class prediction. We then use the trained classifier to predict pseudo labels for the target domain unla- belled data. Finally, we learn a projec- tion for the target domain as we did for the source domain using the pseudo-labelled target domain data, where we maximise the distance between nearest neighbours having opposite pseudo labels. Experi- ments on a standard benchmark dataset for domain adaptation show that the proposed method consistently outperforms numer- ous baselines and returns competitive re- sults comparable to that of SOTA includ- ing self-training, tri-training, and neural adaptations. 1 Introduction A machine learning model trained using data from one domain (source domain) might not necessarily perform well on a different (target) domain when their distributions are different. Domain adapta- tion (DA) considers the problem of adapting a ma- chine learning model such as a classifier that is trained using a source domain to a target domain. In particular, in Unsupervised Domain Adaptation (UDA) (Blitzer et al., 2006, 2007; Pan et al., 2010) we do not assume the availability of any labelled instances from the target domain but a set of la- belled instances from the source domain and un- labelled instances from both source and the target domains. Two main approaches for UDA can be identi- fied from prior work: projection-based and self- training. Projection1-based methods for UDA learn an embedding space where the distribution of fea- tures in the source and the target domains become closer to each other than they were in the origi- nal feature spaces (Blitzer et al., 2006). For this purpose, the union of the source and target feature spaces is split into domain-independent (often re- ferred to as pivots) and domain-specific features using heuristics such as mutual information or fre- quency of a feature in a domain. A projection is then learnt between those two feature spaces and used to adapt a classifier trained from the source domain labelled data. For example, meth- ods based on different approaches such as graph- decomposition spectral feature alignment (Pan et al., 2010) or autoencoders (Louizos et al., 2015) have been proposed for this purpose. Self-training (Yarowsky, 1995; Abney, 2007) is a technique to iteratively increase a set of labelled instances …

Content truncated to 500 words