Error-driven HMM-based Chunk Tagger with Context-dependent Lexicon

Error-driven HMM-based Chunk Tagger with Context-dependent Lexicon GuoDong ZHOU Kent Ridge Digital Labs 21 Heng Hui Keng Terrace Singapore 119613 zhougd@krdl.org.sg Jian SU Kent Ridge Digital Labs 21 Heng Hui Keng Terrace Singapore 119613 sujian@krdl.org.sg Abstract This paper proposes a new error-driven HMM- based text chunk tagger with context-dependent lexicon. Compared with standard HMM-based tagger, this tagger uses a new Hidden Markov Modelling approach which incorporates more contextual information into a lexical entry. Moreover, an error-driven learning approach is adopted to decrease the memory requirement by keeping only positive lexical entries and makes it possible to further incorporate more context- dependent lexical entries. Experiments show that this technique achieves overall precision and recall rates of 93.40% and 93.95% for all chunk types, 93.60% and 94.64% for noun phrases, and 94.64% and 94.75% for verb phrases when trained on PENN WSJ TreeBank section 00-19 and tested on section 20-24, while 25-fold validation experiments of PENN WSJ TreeBank show overall precision and recall rates of 96.40% and 96.47% for all chunk types, 96.49% and 96.99% for noun phrases, and 97.13% and 97.36% for verb phrases. Introduction Text chunking is to divide sentences into non- overlapping segments on the basis of fairly superficial analysis. Abney(1991) proposed this as a useful and relatively tractable precursor to full parsing, since it provides a foundation for further levels of analysis, while still allowing more complex attachment decisions to be postponed to a later phase. Text chunking typically relies on fairly simple and efficient processing algorithms. Recently, many researchers have looked at text chunking in two different ways: Some researchers have applied rule-based methods, combining lexical data with finite state or other rule constraints, while others have worked on inducing statistical models either directly from the words and/or from automatically assigned part-of-speech classes. On the statistics-based approaches, Skut and Brants(1998) proposed a HMM-based approach to recognise the syntactic structures of limited length. Buchholz, Veenstra and Daelemans(1999), and Veenstra(1999) explored memory-based learning method to fred labelled chunks. Ratnaparkhi(1998) used maximum entropy to recognise arbitrary chunk as part of a tagging task. On the rule-based approaches, Bourigaut(1992) used some heuristics and a grammar to extract "terminology noun phrases" from French text. Voutilainen(1993) used similar method to detect English noun phrases. Kupiec(1993) applied. finite state transducer in his noun phrases recogniser for both English and French. Ramshaw and Marcus(1995) used transformation-based learning, an error-driven learning technique introduced by Eric Bn11(1993), to locate chunks in the tagged corpus. Grefenstette(1996) applied finite state transducers to fred noun phrases and verb phrases. In this paper, we will focus on statistics- based methods. The structure of this paper is as follows: In section 1, we will briefly describe the new error-driven HMM-based chunk tagger with context-dependent lexicon in principle. In section 2, a baseline system which only includes the current part-of-speech in the lexicon is given. In section 3, several extended systems with different context-dependent lexicons are described. In section 4, an error=driven learning method is used to decrease memory requirement of the lexicon …

Content truncated to 500 words