Unsupervised Learning of Disambiguation Rules for Part of Speech Tagging

Unsupervised Learning of Disambiguation Rules for Part of Speech Tagging Eric BrilP Department of Computer Science Johns Hopkins University brill@cs, j hu. edu Abstract In this paper we describe an unsupervised learning algorithm for automatically training a rule-based part of speech tagger without using a manually tagged corpus. We compare this algorithm to the Baum-Welch algorithm, used for unsupervised training of stochastic taggers. Next, we show a method for combining unsupervised and supervised rule-based training algorithms to create a highly accurate tagger using only a small amount of manually tagged text. Introduction There has recently been a great deal of work exploring methods for automatically training part of speech taggers, as an alternative to laboriously hand-crafting rules for tagging, as was done in the past [Klein and Simmons, 1963; Harris, 1962]. Almost all of the work in the area of automatically trained taggers has explored Markov-model based part of speech tagging [Jelinek, 1985; Church, 1988; Derose, 1988; DeMarcken, 1990; Cutting et al., 1992; Kupiec, 1992; Charniak et al., 1993; Weischedel et al., 1993; Schutze and Singer, 1994; Lin et al., 1994; Elworthy, 1994; Merialdo, 1995]. 2 For a Markov-model based tagger, training consists of learning both lexical probabilities (P(wordltag)) and contextual probabilities (P(tagiltagi_l ... tagi-n)). Once trained, a sentence can be tagged by searching for the tag sequence that maximizes the product of lexical and contextual probabilities. The most accurate stochastic taggers use estimates of lexical and contextual proba- bilities extracted from large manually annotated corpora (eg. [Weischedel et al., 1993; Charniak et al., 1993]). It is possible to use unsupervised learning to train stochastic taggers without the need for a manually annotated corpus by using the Baum-Welch al- gorithm [Baum, 1972; Jelinek, 1985; Cutting et al., 1992; Kupiec, 1992; Elworthy, 1994; Merialdo, 1995]. This algorithm works by iteratively adjusting the lexical and contextual probabilities to increase the overall probability of the training corpus. If no prior knowledge is available, probabilities are initially either assigned randomly or evenly distributed. Al- though less accurate than the taggers built using manually annotated corpora, the fact that they can be trained using only a dictionary listing the allowable parts of speech for each word and not needing a manually tagged corpus is a huge advantage in many situations. Al- though a number of manually tagged corpora are available (eg. [Francis and Kucera, 1982; Marcus et al., 1993]), training on a corpus of one type and then applying the tagger to a corpus of a different type usually results in a tagger with low accuracy [Weischedel et al., 1993]. Therefore, if tagged text is needed in training, this would require manually tagging 1This work was funded in part by NSF grant IRI-9502312. 2Some other approaches to tagging are described in [Hindle, 1989; Black et al., 1992]. text each time the tagger is to be apphed to a new language, and even when being applied to a new type of text. In [Brill, 1992; Brill, 1994], a rule-based part of speech tagger is described which …

Content truncated to 500 words