Towards Robust Cross-Domain Domain Adaptation for Part-of-Speech Tagging

International Joint Conference on Natural Language Processing, pages 198–206, Nagoya, Japan, 14-18 October 2013. Towards Robust Cross-Domain Domain Adaptation for Part-of-Speech Tagging Tobias Schnabel CIS, University of Munich tbs49@cornell.edu Hinrich Schütze CIS, University of Munich inquiries@cislmu.org Abstract We investigate the robustness of domain adaptation (DA) representations and meth- ods across target domains using part-of- speech (POS) tagging as a case study. We find that there is no single representation and method that works equally well for all target domains. In particular, there are large differences between target domains that are more similar to the source domain and those that are less similar. 1 Introduction Domain adaptation (DA) is the problem of adapt- ing a statistical classifier that was trained on a source domain (SD) to a target domain (TD) for which no or little training data is available. We present a case study that investigates the robust- ness of DA across six different TDs for POS tag- ging. Most prior work on DA has either been on a single TD, on two or more tasks – which results in an experimental setup in which two variables change at the same time, task and TD – or has not systematically investigated how robust differ- ent features and different DA approaches are. The two main information sources in POS tag- ging are context – which POS’s are possible in a particular syntactic context – and lexical bias – the prior probability distribution of POS’s for each word. We address DA for lexical bias in this pa- per, focusing on unknown words; they are most difficult to handle in DA because no direct infor- mation about their possible POS is available in the SD training set. Since typical TDs contain a high percentage of unknown words, a substantial gain in the overall performance can be achieved by im- proving tagging for these words. We address a problem setting where – in addi- tion to labeled SD data – a large amount of un- labeled TD data is available, but no labeled TD data. This setting is often called unsupervised do- main adaptation (cf. (Daumé III, 2007)). We make three contributions in this paper. First, we systematically investigate the cross-TD robust- ness of different representations and methods. We show that there are some elements of DA setups used in the literature that are robust across TDs – e.g., the use of distributional information – but that many others are not, including dimensionality reduction and shape information. Second, we present an analysis that shows that there are two important factors that influence cross-TD variation: (i) the magnitude of the differ- ence in distributional properties between SD and TD – more similar TDs require other methods than less similar TDs and (ii) the evaluation measures used for performance. Since in unsupervised DA we optimize learning criteria on a SD that can be quite different from the TD, different TD evalua- tion measures can diverge more in DA than in stan- dard supervised learning settings when comparing …

Content truncated to 500 words