Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging

Transactions of the Association for Computational Linguistics, 1 (2013) 1–12. Action Editor: Sharon Goldwater. Submitted 11/2012; Revised 1/2013; Published 3/2013. c⃝2013 Association for Computational Linguistics. Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging Oscar T¨ackstr¨om⋄†∗ Dipanjan Das‡ Slav Petrov‡ Ryan McDonald‡ Joakim Nivre†∗ ⋄ Swedish Institute of Computer Science † Department of Linguistics and Philology, Uppsala University ‡ Google Research, New York oscar@sics.se {dipanjand|slav|ryanmcd}@google.com joakim.nivre@lingfil.uu.se Abstract We consider the construction of part-of-speech taggers for resource-poor languages. Recently, manually constructed tag dictionaries from Wiktionary and dictionaries projected via bitext have been used as type constraints to overcome the scarcity of annotated data in this setting. In this paper, we show that additional token constraints can be projected from a resource- rich source language to a resource-poor target language via word-aligned bitext. We present several models to this end; in particular a par- tially observed conditional random field model, where coupled token and type constraints pro- vide a partial signal for training. Averaged across eight previously studied Indo-European languages, our model achieves a 25% relative error reduction over the prior state of the art. We further present successful results on seven additional languages from different families, empirically demonstrating the applicability of coupled token and type constraints across a diverse set of languages. 1 Introduction Supervised part-of-speech (POS) taggers are avail- able for more than twenty languages and achieve ac- curacies of around 95% on in-domain data (Petrov et al., 2012). Thanks to their efficiency and robustness, supervised taggers are routinely employed in many natural language processing applications, such as syn- tactic and semantic parsing, named-entity recognition and machine translation. Unfortunately, the resources required to train supervised taggers are expensive to create and unlikely to exist for the majority of written ∗Work primarily carried out while at Google Research. languages. The necessity of building NLP tools for these resource-poor languages has been part of the motivation for research on unsupervised learning of POS taggers (Christodoulopoulos et al., 2010). In this paper, we instead take a weakly supervised approach towards this problem. Recently, learning POS taggers with type-level tag dictionary constraints has gained popularity. Tag dictionaries, noisily pro- jected via word-aligned bitext, have bridged the gap between purely unsupervised and fully supervised taggers, resulting in an average accuracy of over 83% on a benchmark of eight Indo-European languages (Das and Petrov, 2011). Li et al. (2012) further im- proved upon this result by employing Wiktionary1 as a tag dictionary source, resulting in the hitherto best published result of almost 85% on the same setup. Although the aforementioned weakly supervised approaches have resulted in significant improvements over fully unsupervised approaches, they have not exploited the benefits of token-level cross-lingual projection methods, which are possible with word- aligned bitext between a target language of interest and a resource-rich source language, such as English. This is the setting we consider in this paper (§2). While prior work has successfully considered both token- and type-level projection across word-aligned bitext for estimating the model parameters of genera- tive tagging …

Content truncated to 500 words