Leveraging Auxiliary Tasks for Document-Level Cross-Domain Sentiment Classification

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 654–663, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Leveraging Auxiliary Tasks for Document-Level Cross-Domain Sentiment Classification Jianfei Yu School of Information Systems Singapore Management University jfyu.2014@phdis.smu.edu.sg Jing Jiang School of Information Systems Singapore Management University jingjiang@smu.edu.sg Abstract In this paper, we study domain adap- tation with a state-of-the-art hierarchical neural network for document-level senti- ment classification. We first design a new auxiliary task based on sentiment scores of domain-independent words. We then propose two neural network architectures to respectively induce document embed- dings and sentence embeddings that work well for different domains. When these document and sentence embeddings are used for sentiment classification, we find that with both pseudo and external senti- ment lexicons, our proposed methods can perform similarly to or better than sev- eral highly competitive domain adaptation methods on a benchmark dataset of prod- uct reviews. 1 Introduction Sentiment classification is a fundamental task in opinion mining (Pang et al., 2002; Hu and Liu, 2004; Choi and Cardie, 2008; Nakagawa et al., 2010). Recently, with the advances of deep learn- ing techniques for many NLP applications, various kinds of neural network (NN)-based models have been proposed for this task (Socher et al., 2013; Lei et al., 2015; Yang et al., 2016). As with any supervised learning method, the NN-based models also suffer from the domain adaptation problem, where training data and test data come from different domains. The reason for this is that sentiments are often expressed with domain-specific words and expressions. For ex- ample, in the Book domain, expressions like an insider’s look and a must read are usually posi- tive, but they may not be useful for the Kitchen domain. Similarly, words such as sharp and clean, which are positive in the Kitchen domain, can rarely be seen in the Book domain. Due to the high cost of obtaining labeled data, it would be very attractive if we can adapt a model trained on a source domain to a target domain. A number of different models have been pro- posed for cross-domain sentiment classification, and the core idea of them is to learn a shared la- tent representation that is general across domains. Most of these studies can be categorized into two lines. The first line of work focuses on carefully designing some auxiliary prediction tasks to in- duce a robust cross-domain representation (Blitzer et al., 2007; Pan et al., 2010; Bollegala et al., 2015, 2016). With the trend of deep learning, another line of work centers on employing denoising auto- encoders to learn hidden representations across domains in a purely unsupervised learning man- ner (Glorot et al., 2011; Chen et al., 2012; Zhou et al., 2016). However, most of the two lines of research are based on traditional discrete feature representa- tions, and the induced shared representations are not necessarily specific to sentiment classification. In our recent work, we designed two simple aux- iliary tasks, which are closely …

Content truncated to 500 words