Hybrid Method of Semi-supervised Learning and Feature Weighted Learning for Domain Adaptation of Document Classification

Hybrid Method of Semi-supervised Learning and Feature Weighted Learning for Domain Adaptation of Document Classification Hiroyuki Shinnou, Liying Xiao, Minoru Sasaki, Kanako Komiya Ibaraki University, Department of Computer and Information Sciences 4-12-1 Nakanarusawa, Hitachi, Ibaraki JAPAN 316-8511 hiroyuki.shinnou.0828@vc.ibaraki.ac.jp, {14nm721x, minoru.sasaki.01, kanako.komiya.nlp} @vc.ibaraki.ac.jp Abstract In regard to document classification, semi- supervised learning using the Naive Bayes method and EM algorithm was a great suc- cess, and we refer to this method as NBEM in this paper. Although NBEM is also effec- tive for domain adaption of document classifi- cation, there is still room for improvement be- cause NBEM does not employ valuable infor- mation for this task, that is the difference be- tween source domain and target domain. Here, according to the similarity between the label distribution of the feature on source domain and the estimated label distribution of the fea- ture on target domain, we set the weight on the features to reconstruct the training data. We use this reconstructed training data to per- form document classification by NBEM. As a result of experiment by using a part of 20 Newsgroups, the effect of this method was confirmed. 1 Introduction In this paper, for the domain adaption problems of document classification, we propose a hybrid method of semi-supervised learning and feature weighted learning. In many of the tasks of natural language processing, supervised learning has been a great success. However, if we want to use a su- pervised learning for real problems, there is often problems in domain adaptation. In general, the su- pervised learning is used to create a classifier which is usually using a learning algorithm such as support vector machine (SVM) by labeled training data, then it is possible to identify the label of the test data us- ing this classifier. In this case, the problem is that the domain of training data and test data is different, so it is a problem of domain adaptation (Søgaard, 2013). As a typical example, there is a sentiment analy- sis task to judge whether a review article for a com- modity is positive or not (Blitzer et al., 2007). For example, if we use review articles for ”book” as the training data to make a classifier, the classifier can not correctly identify the review articles for ”movie” which is in another domain. In addition to the emo- tion analysis, supervised learning such as morpho- logical analysis (Mori, 2012), parsing (Sagae and Tsujii, 2007), word sense disambiguation (Shinnou et al., 2015) (Komiya and Okumura, 2012) (Komiya and Okumura, 2011) is utilized in all tasks, it is pos- sible that the domain adaptation problems come into being. In general, the method of the domain adapta- tion can be divided into instance-based method and feature-based method (Pan and Yang, 2010). Instance-based method is a method of learning us- ing weighted training data. Learning under covari- ate shift (Sugiyama and Kawanabe, 2011) is typ- ical in this method. The covariate shift means the assumption that PS(x) ̸= PT (x), PS(y|x) = PT (y|x). …

Content truncated to 500 words