FLORS: Fast and Simple Domain Adaptation for Part-of-Speech Tagging

FLORS: Fast and Simple Domain Adaptation for Part-of-Speech Tagging Tobias Schnabel Department of Computer Science Cornell University tbs49@cornell.edu Hinrich Schütze Center for Information & Language Processing University of Munich inquiries@cislmu.org Abstract We present FLORS, a new part-of-speech tag- ger for domain adaptation. FLORS uses ro- bust representations that work especially well for unknown words and for known words with unseen tags. FLORS is simpler and faster than previous domain adaptation methods, yet it has significantly better accuracy than several baselines. 1 Introduction In this paper we describe FLORS, a part-of-speech (POS) tagger that is Fast in training and tagging, uses LOcal context only (as opposed to finding the optimal tag sequence for the entire sentence), per- forms Robustly on target domains (TDs) in unsu- pervised domain adaptation (DA) and is Simple in architecture and feature representation. FLORS constructs a robust representation of the local context of the word v that is to be tagged. This representation consists of distributional fea- tures, suffixes and word shapes of v and its local neighbors. We show that it has two advantages. First, since the main predictors used by FLORS are distributional features (not the word’s identity), FLORS predicts unseen tags of known words bet- ter than prior work on DA for POS. Second, since FLORS uses representations computed from unla- beled text, representations of unknown words are in principle of the same type as representations of known words; this property of FLORS results in better performance on unknown words compared to prior work. These two advantages are especially beneficial for TDs that contain high rates of unseen tags of known words and high rates of unknown words. We show that FLORS achieves excellent DA tagging results on the five domains of the SANCL 2012 shared task (Petrov and McDonald, 2012) and outperforms three state-of-the-art taggers on Blitzer et al.’s (2006) biomedical data. FLORS is also simpler and faster than other POS DA methods. It is simple in that the input repre- sentation consists of three simple types of features: distributional count features and two types of binary features, suffix and shape features. Many other word representations that are used for improving general- ization (e.g., (Brown et al., 1992; Collobert et al., 2011)) are costly to train or have difficulty han- dling unknown words. Our representations are fast to build and can be created on-the-fly for unknown words that occur during testing. The learning architecture is simple and fast as well. We train k binary one-vs-all classifiers that use local context only and no sequence informa- tion (where k is the number of tags). Thus, tag- ging complexity is O(k). Many other learning se- tups for DA are more complex; e.g., they learn rep- resentations (as opposed to just counting), they learn several classifiers for different subclasses of words (e.g., known vs. unknown) or they combine left-to- right and right-to-left taggings. The next two sections describe experimental data, setup and results. Results are discussed in Section 4. We compare FLORS to alternative word representa- …

Content truncated to 500 words