A Report of Recent Progress in Transformation-Based Error-Driven Learning

A Report of Recent Progress in Transformation-Based Error-Driven Learning* Eric Brill ABSTRACT Most recent research in trainable part of speech taggers has explored stochastic tagging. While these taggers obtain high accuracy, linguistic information is captured indirectly, typi- cally in tens of thousands of lexical and contextual probabili- ties. In [Brill 92], a trainable rule-based tagger was described that obtained performance comparable to that of stochas- tic taggers, but captured relevant linguistic information in a sma]_l number of simple non-stochastic rules. In this pa- per, we describe a number of extensions to this rule-based tagger. First, we describe a method for expressing lexical re- lations in tagging that stochastic taggers are currently unable to express. Next, we show a rule-based approach to tagging unknown words. Finally, we show how the tagger can be extended into a k-best tagger, where multiple tags can be assigned to words in some cases of uncertainty. Spoken Language Systems Group Laboratory for Computer Science Massachusetts Institute of Technology Cambridge, Massachusetts 02139 / that achieves performance comparable to that of stochas- tic taggers. Training this tagger is fully automated, but unlike trainable stochastic taggers, linguistic information is encoded directly in a set of simple non-stochastic rules. In this paper, we describe some extensions to this rule- based tagger. These include a rule-based approach to: lexicalizing the tagger, tagging unknown words, and as- signing the k-best tags to a word. All of these extensions, as well as the original tagger, are based upon a learn- ing paradigm called transformation-based error-driven learning. This learning paradigm has shown promise in a number of other areas of natural language processing, and we hope that the extensions to transformation-based learning described in this paper can carry over to other domains of application as well. 2 1. INTRODUCTION When automated part of speech tagging was initially ex- plored [Klein and Simmons 63, Harris 62], people manu- ally engineered rules for tagging, sometimes with the aid of a corpus. As large corpora became available, it be- came clear that simple Markov-model based stochastic taggers that were automatically trained could achieve high rates of tagging accuracy [Jelinek 85]. These stochastic taggers have a number of advantages over the manually built taggers, including obviating the need for laborious manual rule construction, and possibly captur- ing useful information that may not have been noticed by the human engineer. However, stochastic taggers have the disadvantage that linguistic information is only cap- tured indirectly, in large tables of statistics. Almost all recent work in developing automatically trained part of speech taggers has been on further exploring Markov- model based tagging [Jetinek 85, Church 88, DeRose 88, DeMarcken 90, Merialdo 91, Cutting et al. 92, Kupiec 92, Charniak et al. 93, Weischedel et al. 93]. 1 In [Brill 92], a trainable rule-based tagger is described *This research was supported by ARPA under contract N00014- 89-J-1332, monitored through the Office of Naval Research. 1Markov-model based taggers assign a sentence the tag sequence that maximizes Prob(word[tag) * Prob(taglprevious n tags). 256 2. …

Content truncated to 500 words