Neural Structural Correspondence Learning for Domain Adaptation

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 400–410, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics Neural Structural Correspondence Learning for Domain Adaptation Yftah Ziser and Roi Reichart Faculty of Industrial Engineering and Management, Technion, IIT syftah@campus.technion.ac.il, roiri@ie.technion.ac.il Abstract We introduce a neural network model that marries together ideas from two prominent strands of research on domain adaptation through representation learning: structural correspondence learning (SCL, (Blitzer et al., 2006)) and autoencoder neural net- works (NNs). Our model is a three-layer NN that learns to encode the non-pivot features of an input example into a low- dimensional representation, so that the ex- istence of pivot features (features that are prominent in both domains and convey useful information for the NLP task) in the example can be decoded from that repre- sentation. The low-dimensional represen- tation is then employed in a learning al- gorithm for the task. Moreover, we show how to inject pre-trained word embed- dings into our model in order to improve generalization across examples with simi- lar pivot features. We experiment with the task of cross-domain sentiment classifica- tion on 16 domain pairs and show substan- tial improvements over strong baselines.1 1 Introduction Many state-of-the-art algorithms for Natural Lan- guage Processing (NLP) tasks require labeled data. Unfortunately, annotating sufficient amounts of such data is often costly and labor intensive. Consequently, for many NLP applications even resource-rich languages like English have labeled data in only a handful of domains. Domain adaptation (Daum´e III, 2007; Ben- David et al., 2010), training an algorithm on labeled data taken from one domain so that it 1Our code is at: https://github.com/yftah89/Neural-SCL- Domain-Adaptation. can perform properly on data from other do- mains, is therefore recognized as a fundamental challenge in NLP. Indeed, over the last decade domain adaptation methods have been proposed for tasks such as sentiment classification (Bolle- gala et al., 2011b), POS tagging (Schnabel and Sch¨utze, 2013), syntactic parsing (Reichart and Rappoport, 2007; McClosky et al., 2010; Rush et al., 2012) and relation extraction (Jiang and Zhai, 2007; Bollegala et al., 2011a), if to name just a handful of applications and works. Leading recent approaches to domain adapta- tion in NLP are based on Neural Networks (NNs), and particularly on autoencoders (Glorot et al., 2011; Chen et al., 2012). These models are be- lieved to extract features that are robust to cross- domain variations. However, while excelling on benchmark domain adaptation tasks such as cross- domain product sentiment classification (Blitzer et al., 2007), the reasons to this success are not entirely understood. In the pre-NN era, a prominent approach to do- main adaptation in NLP, and particularly in sen- timent classification, has been structural corre- spondence learning (SCL) (Blitzer et al., 2006, 2007). Following the auxiliary problems ap- proach to semi-supervised learning (Ando and Zhang, 2005), this method identifies correspon- dences among features from different domains by modeling their correlations with pivot features: features that are frequent in both domains and are …

Content truncated to 500 words