Hybrid Text Chunking

In: Proceedings of CoNLL-2000 and LLL-2000, pages 163-165, Lisbon, Portugal, 2000. Hybrid Text Chunking GuoDong Zhou and Jian Su and TongGuan Tey Kent Ridge Digital Labs 21 Heng Mui Keng Terrace Singapore 119613 {zhougd, sujian, tongguan}@krdl, org.sg Abstract This paper proposes an error-driven HMM- based text chunk tagger with context-dependent lexicon. Compared with standard HMM-based tagger, this tagger incorporates more contextual information into a lexical entry. Moreover, an error-driven learning approach is adopted to de- crease the memory requirement by keeping only positive lexical entries and makes it possible to further incorporate more context-dependent lexical entries. Finally, memory-based learning is adopted to further improve the performance of the chunk tagger. 1 Introduction The idea of using statistics for chunking goes back to Church(1988), who used corpus frequen- cies to determine the boundaries of simple non- recursive noun phrases. Skut and Brants(1998) modified Church's approach in a way permitting efficient and reliable recognition of structures of limited depth and encoded the structure in such a way that it can be recognised by a Viterbi tagger. Our approach follows Skut and Brants' way by employing HMM-based tagging method to model the chunking process. 2 HMM-based Chunk Tagger with Context-dependent Lexicon Given a token sequence G~ = glg2""gn , the goal is to find an optimal tag sequence T~ = tit2.. "tn which maximizes log P(T~IG~): . P(T~,G?) log P(T~IG?) = log P(T~) +log p(T~)P(G?) The second item in the above equation is the mutual information between the tag sequence T~ and the given token sequence G~. By as- suming that the mutual information between G~ and T~ is equal to the summation off mutual information between G~ and the individual tag ti (l<i_<n): l P(T~, G~) n _ P(ti, G?) o g p - ~ ) = E log P'(t~-P-~) i=1 n MI(T~, G?) = ~ MI(ti, G~), i=1 we have: n P(ti, G~) log P(T~IG~) = log P(T~)+~ log P(~i)P-~) i=1 n n = log P(T~)- ~ log P(ti) + ~ log P(tilG?) i----1 i=1 The first item of above equation can be solved by chain rules. Normally, each tag is assumed to be probabilistic dependent on the N-1 previ- ous tags. Here, backoff bigram(N=2) model is used. The second item is the summation of log probabilities of all the tags. Both the first item and second item constitute the language model component while the third item constitutes the lexicon component. Ideally the third item can be estimated by the forward-backward algo- rithm(Rabiner 1989) recursively for the first- order(Rabiner 1989) or second-order HMMs. However, several approximations on it will be attempted later in this paper instead. The stochastic optimal tag sequence can be found by maximizing the above equation over all the possible tag sequences using the Viterbi algo- rithm. The main difference between our tagger and the standard taggers lies in our tagger has a context-dependent lexicon while others use a context-independent lexicon. 163 For chunk tagger, we have gl = piwi where W~ = wlw2""Wn is the word sequence …

Content truncated to 500 words