If You Even Don’t Have a Bit of Bible: Learning Delexicalized POS Taggers

If You Even Don’t Have a Bit of Bible: Learning Delexicalized POS Taggers Zhiwei Yu,† David Mareˇcek,⋆ Zdenˇek ˇZabokrtsk´y,⋆ Daniel Zeman⋆ †Department of Computer Science and Engineering, Shanghai Jiaotong University ⋆Institute of Formal and Applied Linguistics, Faculty of Mathematics and Physics, Charles University in Prague jordanyzw@sjtu.edu.cn, {marecek,zabokrtsky,zeman}@ufal.mff.cuni.cz Abstract Part-of-speech (POS) induction is one of the most popular tasks in research on unsupervised NLP. Various unsupervised and semi- supervised methods have been proposed to tag an unseen language. However, many of them require some partial understanding of the target language because they rely on dictionaries or parallel corpora such as the Bible. In this paper, we propose a different method named delexicalized tagging, for which we only need a raw corpus of the target language. We transfer tagging models trained on annotated corpora of one or more resource-rich languages. We employ language-independent features such as word length, frequency, neighborhood entropy, character classes (alphabetic vs. numeric vs. punctuation) etc. We demonstrate that such features can, to certain extent, serve as predictors of the part of speech, represented by the universal POS tag (Das and Petrov, 2011). Keywords: delexicalized tagging, HamdleDT 2.0, features expansion, classifier 1. Introduction Part-of-speech (POS) tagging is sometimes considered an almost solved problem in NLP. Standard supervised ap- proaches often reach accuracy above 95% if sufficiently large hand-labeled training data are available (typically sev- eral hundred thousand tokens or more). However, we still believe that it makes sense to study semi-supervised and unsupervised approaches because of the following reasons: • It is hardly realistic to expect that manual annotation efforts will be ever invested into all 7,000 languages. • Even if it might be very efficient to start at least with some small annotated data, we believe that adding new features independent of hand-tagged text might be helpful in a combination of supervised and unsu- pervised methods, e.g., for better handling of out-of- vocabulary words. • We should keep in mind that the “standard” POS distinctions—although broadly used—are not mani- fested in languages directly. They result from certain linguistic tradition whose current dominance can be attributed to geopolitical reasons rather than its lin- guistic “obviousness”. Thus, for instance, if we say that something is an adverb in language X, we should be able to support such a claim by some measurable evidence rather than just by saying that it becomes an adverb if translated to English. • For some multilingual NLP tasks, such as unsuper- vised dependency parsing (or parser transfer), it might be more important to preprocess all languages un- der study as similarly as possible (including POS tag- ging), rather than to maximize accuracy with respect to highly different gold-standard data in individual languages. We propose “delexicalized tagging”, a new method for under-resourced languages. In analogy to delexical- ized parsing (Zeman and Resnik, 2008), we transfer a tagging model from a resource-rich language (or a set of languages); the model is independent of individual word forms. In delexicalized parsing, word form sequences are substituted by sequences of …

Content truncated to 500 words