Weakly Supervised Part-of-Speech Tagging for Morphologically-Rich, Resource-Scarce Languages

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 363–371, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics Weakly Supervised Part-of-Speech Tagging for Morphologically-Rich, Resource-Scarce Languages Kazi Saidul Hasan and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {saidul,vince}@hlt.utdallas.edu Abstract This paper examines unsupervised ap- proaches to part-of-speech (POS) tagging for morphologically-rich, resource-scarce languages, with an emphasis on Goldwa- ter and Griffiths’s (2007) fully-Bayesian approach originally developed for En- glish POS tagging. We argue that ex- isting unsupervised POS taggers unreal- istically assume as input a perfect POS lexicon, and consequently, we propose a weakly supervised fully-Bayesian ap- proach to POS tagging, which relaxes the unrealistic assumption by automatically acquiring the lexicon from a small amount of POS-tagged data. Since such relaxation comes at the expense of a drop in tag- ging accuracy, we propose two extensions to the Bayesian framework and demon- strate that they are effective in improv- ing a fully-Bayesian POS tagger for Ben- gali, our representative morphologically- rich, resource-scarce language. 1 Introduction Unsupervised POS tagging requires neither man- ual encoding of tagging heuristics nor the avail- ability of data labeled with POS information. Rather, an unsupervised POS tagger operates by only assuming as input a POS lexicon, which con- sists of a list of possible POS tags for each word. As we can see from the partial POS lexicon for English in Figure 1, “the” is unambiguous with re- spect to POS tagging, since it can only be a deter- miner (DT), whereas “sting” is ambiguous, since it can be a common noun (NN), a proper noun (NNP) or a verb (VB). In other words, the lexi- con imposes constraints on the possible POS tags Word POS tag(s) ... ... running NN, JJ sting NN, NNP, VB the DT ... ... Figure 1: A partial lexicon for English of each word, and such constraints are then used by an unsupervised tagger to label a new sentence. Conceivably, tagging accuracy decreases with the increase in ambiguity: unambiguous words such as “the” will always be tagged correctly; on the other hand, unseen words (or words not present in the POS lexicon) are among the most ambigu- ous words, since they are not constrained at all and therefore can receive any of the POS tags. Hence, unsupervised POS tagging can present sig- nificant challenges to natural language processing researchers, especially when a large fraction of the words are ambiguous. Nevertheless, the de- velopment of unsupervised taggers potentially al- lows POS tagging technologies to be applied to a substantially larger number of natural languages, most of which are resource-scarce and, in particu- lar, have little or no POS-tagged data. The most common approach to unsupervised POS tagging to date has been to train a hidden Markov model (HMM) in an unsupervised man- ner to maximize the likelihood of an unannotated corpus, using a special instance of the expectation- maximization (EM) algorithm (Dempster et al., …

Content truncated to 500 words