Learning a Part-of-Speech Tagger from Two Hours of Annotation

Proceedings of NAACL-HLT 2013, pages 138–147, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Learning a Part-of-Speech Tagger from Two Hours of Annotation Dan Garrette Department of Computer Science The University of Texas at Austin dhg@cs.utexas.edu Jason Baldridge Department of Linguistics The University of Texas at Austin jbaldrid@utexas.edu Abstract Most work on weakly-supervised learning for part-of-speech taggers has been based on un- realistic assumptions about the amount and quality of training data. For this paper, we attempt to create true low-resource scenarios by allowing a linguist just two hours to anno- tate data and evaluating on the languages Kin- yarwanda and Malagasy. Given these severely limited amounts of either type supervision (tag dictionaries) or token supervision (labeled sentences), we are able to dramatically im- prove the learning of a hidden Markov model through our method of automatically general- izing the annotations, reducing noise, and in- ducing word-tag frequency information. 1 Introduction The high performance achieved by part-of-speech (POS) taggers trained on plentiful amounts of la- beled word tokens is a success story of computa- tional linguistics (Manning, 2011). However, re- search on learning taggers using type supervision (e.g. tag dictionaries or morphological transducers) has had a more checkered history. The setting is a seductive one: by labeling the possible parts-of- speech for high frequency words, one might learn accurate taggers by incorporating the type informa- tion as constraints to a semi-supervised generative learning model like a hidden Markov model (HMM). Early work showed much promise for this strategy (Kupiec, 1992; Merialdo, 1994), but successive ef- forts in recent years have continued to peel away and address layers of unrealistic assumptions about the size, coverage, and quality of the tag dictionaries that had been used (Toutanova and Johnson, 2008; Ravi and Knight, 2009; Hasan and Ng, 2009; Gar- rette and Baldridge, 2012). This paper attempts to strip away further layers so we can build better intu- itions about the effectiveness of type-supervised and token-supervised strategies in a realistic setting of POS-tagging for low-resource languages. In most previous work, tag dictionaries are ex- tracted from a corpus of annotated tokens. To ex- plore the type-supervised scenario, these have been used as a proxy for dictionaries produced by lin- guists. However, this overstates their effectiveness. Researchers have often manually pruned tag dictio- naries by removing low-frequency word/tag pairs; this violates the assumption that frequency informa- tion is not available. Others have also created tag dictionaries by extracting every word/tag pair in a large, labeled corpus, including the test data—even though actual applications would never have such complete lexical knowledge. Dictionaries extracted from corpora are also biased towards including only the most likely tag for each word type, resulting in a cleaner dictionary than one would find in real sce- nario. Finally, tag dictionaries extracted from anno- tated tokens benefit from the annotation process of labeling and review and refinement over an extended collaboration period. Such high quality annotations are simply not available for most low-resource lan- guages. This paper describes an …

Content truncated to 500 words