Domain Adaptation Using a Combination of Multiple Embeddings for Sentiment Analysis

Domain Adaptation Using a Combination of Multiple Embeddings for Sentiment Analysis Hiroyuki Shinnou Xinyu Zhao Kanako Komiya Ibaraki University, Department of Computer and Information Sciences 4-12-1 Nakanarusawa, Hitachi, Ibaraki JAPAN 316-8511 {hiroyuki.shinnou.0828, 17nm711a, kanako.komiya.nlp} @vc.ibaraki.ac.jp Abstract We propose a new method for domain adapta- tion by using a combination of multiple em- beddings for sentiment analysis. We first make the following embeddings for the doc- ument: (1) vector construction by using the bag-of-words model, (2) vector by dimen- sion reduction using SVD, and (3) embedded vector by using doc2vec. We then connect the three embeddings. This connected vec- tor is used as the feature vector in the learn- ing and testing stages. In the experiment, we used an Amazon dataset that has three domains (“books”, “DVD” and “music”) and six types of domain adaptations. The experi- ment showed the effectiveness of our proposed method. 1 Introduction Supervised learning has achieved great success In many natural language processing tasks. However, in real applications, the source domain S in the learning stage may be different from the target do- main T in the testing stage, In this case, the clas- sifier learned in the source domain S is ineffective in the target domain T. For example, in sentiment analysis, the classifier learned through reviews on “books” was ineffective for judging whether a re- view on “DVD” is positive or negative. This prob- lem is known as the “domain shift” problem. In re- cent years, research on domain adaptation has been very active to solve this problem (Søgaard, 2013). There are various types of problems in domain adaptation. However, in the case of domain adap- tation of sentiment analysis, the method that maps data in domains S and T to the shared feature sub- space W is very effective. When we conduct learning and testing on the space W, a domain shift does not occur. There- fore, the manner in which to construct W becomes the problem. For this problem, the representative research is maximum mean discrepancies (MMD) (Borgwardt et al., 2006). In recent years, many methods using deep learning have been proposed (Patel et al., 2015). This study focuses on the sentiment analysis task, wherein the data is the document and W is the space of embeddings for the document. Furthermore, there are several methods to embed documents in the low- dimensional space, but there is no single embedding algorithm that can be used to respond to various do- main adaptations from the diversity of domain dif- ferences. Therefore, we combine the embeddings of the document in this study, specifically the com- bination of doc2vec (Lau and Baldwin, 2016) and singular value decomposition (SVD). The document d is converted to the embedding e and g by using doc2vec and SVD respectively. Moreover we use the vector v obtained from the bag-of-words (BOW) model of the document d. Generally, v is used as the feature vector to learn the classifier. In this study, we use three connected vectors, namely [v : e : …

Content truncated to 500 words