Comparing a Linguistic and a Stochastic Tagger

Comparing a Linguistic and a Stochastic Tagger Christer Samuelsson Atro Voutilainen Lucent Technologies Research Unit for Multilingu~l Language Technology Bell Laboratories P.O. Box 4 600 Mountain Ave, Room 2D-339 FIN-00014 University of Helsinki .Murray Hill, NJ 07974, USA Finland christ er©research, bell-labs, tom Afro. Vout ilainen©Helsinki. FI Abstract Concerning different approaches to auto- matic PoS tagging: EngCG-2, a constraint- based morphological tagger, is compared in a double-blind test with a state-of-the-art statistical tagger on a common disambigua- tion task using a common tag set. The ex- periments show that for the same amount of remaining ambiguity, the error rate of the statistical tagger is one order of mag- nitude greater than that of the rule-based one. The two related issues of priming effects compromising the results and dis- agreement between human annotators are also addressed. 1 Introduction There are currently two main methods for auto- matic part-of-speech tagging. The prevailing one uses essentially statistical language models automat- ically derived from usually hand-annotated corpora. These corpus-based models can be represented e.g. as collocational matrices (Garside et al. (eds.) 1987: Church 1988), Hidden Markov models (cf. Cutting et al. 1992), local rules (e.g. Hindle 1989) and neu- ral networks (e.g. Schmid 1994). Taggers using these statistical language models are generally reported to assign the correct and unique tag to 95-97% of words in running text. using tag sets ranging from some dozens to about 130 tags. The less popular approach is based on hand-coded linguistic rules. Pioneering work was done in the 1960"s (e.g. Greene and Rubin 1971). Recently, new interest in the linguistic approach has been shown e.g. in the work of (Karlsson 1990: Voutilainen et al. 1992; Oflazer and Kuru6z 1994: Chanod and Tapanainen 1995: Karlsson et al. (eds.) 1995; Vouti- lainen 1995). The first serious linguistic competitor to data-driven statistical taggers is the English Con- straint Grammar parser. EngCG (cf. Voutilainen et al. 1992; Karlsson et al. (eds.) 1995). The tagger consists of the following sequentially applied mod- ules: 1. Tokenisation 2. Morphological analysis (a) Lexical component (b) Rule-based guesser for unknown words 3. Resolution of morphological ambiguities The tagger uses a two-level morphological anal- yser with a large lexicon and a morphological description that introduces about 180 different ambiguity-forming morphological analyses, as a re- sult of which each word gets 1.7-2.2 different analy- ses on an average. Morphological analyses are as- signed to unknown words with an accurate rule- based 'guesser'. The morphological disambiguator uses constraint rules that discard illegitimate mor- phological analyses on the basis of local or global context conditions. The rules can be grouped as ordered subgrammars: e.g. heuristic subgrammar 2 can be applied for resolving ambiguities left pending by the more "careful' subgrammar 1. Older versions of EngCG (using about 1,150 con- straints) are reported (~butilainen et al. 1992; Vouti- lainen and HeikkiUi 1994; Tapanainen and Vouti- lainen 1994; Voutilainen 1995) to assign a correct analysis to about 99.7% of all words while each word in the output retains 1.04-1.09 alternative analyses on an …

Content truncated to 500 words