Multi-Source Attention for Unsupervised Domain Adaptation

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 873–883 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 873 Multi-Source Attention for Unsupervised Domain Adaptation Xia Cui University of Liverpool Xia.Cui@liverpool.ac.uk Danushka Bollegala∗ University of Liverpool, Amazon danushka@liverpool.ac.uk Abstract We model source-selection in multi-source Un- supervised Domain Adaptation (UDA) as an attention-learning problem, where we learn at- tention over the sources per given target in- stance. We first independently learn source- specific classification models, and a related- ness map between sources and target domains using pseudo-labelled target domain instances. Next, we learn domain-attention scores over the sources for aggregating the predictions of the source-specific models. Experimental re- sults on two cross-domain sentiment classifica- tion datasets show that the proposed method re- ports consistently good performance across do- mains, and at times outperforming more com- plex prior proposals. Moreover, the computed domain-attention scores enable us to find ex- planations for the predictions made by the pro- posed method. 1 1 Introduction Domain adaptation (DA) considers the problem of generalising a model learnt using the data from a particular source domain to a different target do- main (Zhang et al., 2015). Although most DA meth- ods consider adapting to a target domain from a single source domain (Blitzer et al., 2006, 2007; Ganin et al., 2016), often it is difficult to find a suit- able single source to adapt from, and one must con- sider multiple sources. For example, in sentiment classification, each product category is considered as a domain (Blitzer et al., 2006), resulting in a multi-domain adaptation setting. Unsupervised DA (UDA) is a special case of DA where labelled instances are not available for ∗Danushka Bollegala holds concurrent appointments as a Professor at University of Liverpool and as an Amazon Scholar. This paper describes work performed at the Univer- sity of Liverpool and is not associated with Amazon. 1Source code available at https://github.com/ LivNLP/multi-source-attention the target domain. Existing approaches for UDA can be categorised into pivot-based and instance- based methods. Pivots refer to the features common to both source and target domains (Blitzer et al., 2006). Pivot-based single-source domain adapta- tion methods, such as Structural Correspondence Learning (SCL; Blitzer et al., 2006, 2007) and Spec- tral Feature Alignment (SFA; Pan et al., 2010), first select a set of pivots and then project the source and target domain documents into a shared space. Next, a prediction model is learnt in this shared space. However, these methods fail in multi-source set- tings because it is challenging to find pivots across all sources such that a single shared projection can be learnt. Similarly, instance-based methods, such as Stacked Denoising Autoencoders (SDA; Glo- rot et al., 2011) and marginalised SDA (mSDA; Chen et al., 2012) minimise the loss between the original inputs and their reconstructions. Not all of the source domains are appropriate for learning transferable projections for a particular target do- main. Adapting from …

Content truncated to 500 words