A Common Case of Jekyll and Hyde: The Synergistic Effect of Using Divided Source Training Data for Feature Augmentation

International Joint Conference on Natural Language Processing, pages 623–631, Nagoya, Japan, 14-18 October 2013. A Common Case of Jekyll and Hyde: The Synergistic Effect of Using Divided Source Training Data for Feature Augmentation Yan Song City University of Hong Kong 83, Tat Chee Ave., Kowloon Hong Kong, China clksong@gmail.com Fei Xia University of Washington PO Box 354340 Seattle, WA 98195, USA fxia@uw.edu Abstract Feature augmentation is a well-known method for domain adaptation and has been shown to be effective when tested on several NLP tasks (Daume III, 2007). However, a limitation of the method is that it requires labeled data from the target do- main and very often such data is unavail- able. In this paper, we propose to use train- ing data selection to divide the source do- main training data into two parts, pseudo target data (the selected part) and source data (the unselected part), and then ap- ply feature augmentation on the two parts of the training data. This approach has two advantages: first, feature augmenta- tion can be applied even when there is no labeled data from the target domain; sec- ond, the approach can take advantage of all the training data including the part that is not selected by training data selection. We evaluate the approach on Chinese word segmentation and part-of-speech tagging and show that it outperforms the baseline where no feature augmentation is applied. 1 Introduction The goal of domain adaptation is to alleviate the degradation of NLP systems when training and test data are from different domains. There have been many approaches to domain adaptation, and two of well-known ones are feature augmentation and training data selection. Feature augmentation makes three copies of each feature in the original feature set (one for the source domain, one for the target domain, and one for the general domain) so that features appeared in the source and the tar- get domains can be differentiated in case they be- have differently in the two domains; the method has been shown to be effective for several NLP tasks (Daume III, 2007). However, a limitation of the method is that it requires labeled data in the tar- get domain, a condition that is hard to meet when creating labeled data in the target domain is expen- sive and time-consuming. Training data selection addresses the differ- ences between the source and target domains by choosing a subset of the training data in the source domain that is similar to the data in the target do- main. When the amount of source training data is large, this method often provides better perfor- mance than using the entire training data (Moore and Lewis, 2010; Axelrod et al., 2011; Plank and van Noord, 2011; Song et al., 2012). However, when the amount of the training data is small, the selected subset is unlikely to outperform the en- tire training data because the trained model cannot benefit from unselected labeled data. To address the limitations of both methods, we propose to divide the …

Content truncated to 500 words