Part-of-Speech Tagging in Context

Part of Speech Tagging in Context Michele BANKO and Robert C. MOORE Microsoft Research One Microsoft Way Redmond, WA 98052 USA {mbanko, bobmoore}@microsoft.com Abstract We present a new HMM tagger that exploits context on both sides of a word to be tagged, and evaluate it in both the unsupervised and supervised case. Along the way, we present the first comprehensive comparison of unsupervised methods for part-of-speech tagging, noting that published results to date have not been comparable across corpora or lexicons. Observing that the quality of the lexicon greatly impacts the accuracy that can be achieved by the algorithms, we present a method of HMM training that improves accuracy when training of lexical probabilities is unstable. Finally, we show how this new tagger achieves state-of-the-art results in a supervised, non-training intensive framework. 1 Introduction The empiricist revolution in computational linguistics has dramatically shifted the accepted boundary between what kinds of knowledge are best supplied by humans and what kinds are best learned from data, with much of the human- supplied knowledge now being in the form of annotations of data. As we look to the future, we expect that relatively unsupervised methods will grow in applicability, reducing the need for expensive human annotation of data. With respect to part-of-speech tagging, we believe that the way forward from the relatively small number of languages for which we can currently identify parts of speech in context with reasonable accuracy will make use of unsupervised methods that require only an untagged corpus and a lexicon of words and their possible parts of speech. We believe this based on the fact that such lexicons exist for many more languages (in the form of conventional dictionaries) than extensive human-tagged training corpora exist for. Unsupervised part-of-speech tagging, as defined above, has been attempted using a variety of learning algorithms (Brill 1995, Church, 1988, Cutting et. al. 1992, Elworthy, 1994 Kupiec 1992, Merialdo 1991). While this makes unsupervised part-of-speech tagging a relatively well-studied problem, published results to date have not been comparable with respect to the training and test data used, or the lexicons which have been made available to the learners. In this paper, we provide the first comprehensive comparison of methods for unsupervised part-of- speech tagging. In addition, we explore two new ideas for improving tagging accuracy. First, we explore an HMM approach to tagging that uses context on both sides of the word to be tagged, inspired by previous work on building bidirectionality into graphical models (Lafferty et. al. 2001, Toutanova et. al. 2003). Second we describe a method for sequential unsupervised training of tag sequence and lexical probabilities in an HMM, which we observe leads to improved accuracy over simultaneous training with certain types of models. In section 2, we provide a brief description of the methods we evaluate and review published results. Section 3 describes the contextualized variation on HMM tagging that we have explored. In Section 4 we provide a direct comparison of several unsupervised part-of-speech taggers, which is followed by …

Content truncated to 500 words