Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 236–246, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification Jianfei Yu School of Information Systems Singapore Management University jfyu.2014@phdis.smu.edu.sg Jing Jiang School of Information Systems Singapore Management University jingjiang@smu.edu.sg Abstract In this paper, we study cross-domain senti- ment classification with neural network archi- tectures. We borrow the idea from Structural Correspondence Learning and use two auxil- iary tasks to help induce a sentence embedding that supposedly works well across domains for sentiment classification. We also propose to jointly learn this sentence embedding together with the sentiment classifier itself. Experi- ment results demonstrate that our proposed joint model outperforms several state-of-the- art methods on five benchmark datasets. 1 Introduction With the growing need of correctly identifying the sentiments expressed in subjective texts such as product reviews, sentiment classification has re- ceived continuous attention in the NLP community for over a decade (Pang et al., 2002; Pang and Lee, 2004; Hu and Liu, 2004; Choi and Cardie, 2008; Nakagawa et al., 2010). One of the big challenges of sentiment classification is how to adapt a senti- ment classifier trained on one domain to a different new domain. This is because sentiments are often expressed with domain-specific words and expres- sions. For example, in the Movie domain, words such as moving and engaging are usually positive, but they may not be relevant in the Restaurant do- main. Since labeled data is expensive to obtain, it would be very useful if we could adapt a model trained on a source domain to a target domain. Much work has been done in sentiment analysis to address this domain adaptation problem (Blitzer et al., 2007; Pan et al., 2010; Bollegala et al., 2011; Ponomareva and Thelwall, 2012; Bollegala et al., 2016). Among them, an appealing method is the Structural Correspondence Learning (SCL) method (Blitzer et al., 2007), which uses pivot fea- ture prediction tasks to induce a projected feature space that works well for both the source and the tar- get domains. The intuition behind is that these pivot prediction tasks are highly correlated with the orig- inal task. For sentiment classification, Blitzer et al. (2007) first chose pivot words which have high mu- tual information with the sentiment labels, and then set up the pivot prediction tasks to be the predictions of each of these pivot words using the other words. However, the original SCL method is based on traditional discrete feature representations and lin- ear classifiers. In recent years, with the advances of deep learning in NLP, multi-layer neural net- work models such as RNNs and CNNs have been widely used in sentiment classification and achieved good performance (Socher et al., 2013; Dong et al., 2014a; Dong et al., 2014b; Kim, 2014; Tang et al., 2015). In these models, dense, real-valued feature vectors and non-linear classification func- tions are used. By using real-valued word embed- dings pre-trained from a …

Content truncated to 500 words