HMM Specialization with Selective Lexicalization

I HMM Specialization with Selective Lexicalization* Jin-Dong Kim and Sang-Zoo Lee and Hae-Chang Rim Dept. of Computer Science and Engineering, Korea University, Anam-dong, Seongbuk-ku, Seoul 136-701, Korea E-maih {jinlzoolrim}@nlp.korea.ac.kr Abstract We present a technique which complements Hidden Markov Models by incorporating some lexicalized states representing syntactically un- common words. 'Our approach examines the distribution of transitions, selects the uncom- mon words, and makes lexicalized states for the words. We perfor'med a part-of-speech tagging experiment on the Brown corpus to evaluate the resultant language model and discovered that this technique improved the tagging accuracy by 0.21% at the 95% level of confidence. 1 Introduction Hidden Markov 'Models are widely used for statistical language modelling in various fields, e.g., part-of-speech tagging or speech recogni- tion (Rabiner and Juang, 1986). The models are based on Markov assumptions, which make it possible to view the language prediction as a Markov process. 'In general, we make the first- order Markov ass'umptions that the current tag is only dependant on the previous tag and that the current word is only dependant on the cur- rent tag. These are very 'strong' assumptions, so that the first-order Hidden Markov Models have the advantage of drastically reducing the number of its parameters. On the other hand, the assumptions restrict the model from utiliz- ing enough constraints provided by the local context and the resultant model consults only a single category 'as the contex. A lot of effort has been devoted in the past to make up for the insufficient contextual in- formation of the first-order probabilistic model. The second order Hidden Markov Models with " The research underlying this paper was supported t) 3" research grants fl'om Korea Science and Engineering Foundation. appropriate smoothing techniques show better performance than the first order models and is considered a state-of-the-art technique (Meri- aldo, 1994; Brants, 1996). The complexity of the model is however relatively very high con- sidering the small improvement of the perfor- mance. Garside describes IDIOMTAG (Garside et al., 1987) which is a component of a part-of- speech tagging system named CLAWS. ID- IOMTAG serves as a front-end to the tagger and modifies some initially assigned tags in or- der to reduce the amount of ambiguity to be dealt with by the tagger. IDIOMTAG can look at any combination of words and tags, with or without intervening words. By using the IDIOMTAG, CLAWS system improved tag- ging accuracy from 94% to 96-97%. However, the manual-intensive process of producing id- iom tags is very expensive although IDIOMTAG proved fruitful. Kupiec (Kupiec, 1992) describes a technique of augmenting the Hidden Markov Models for part-of-speech tagging by the use of networks. Besides the original states representing each part-of-speech, the network contains additional states to reduce the noun/adjective confusion, and to extend the context for predicting past participles from preceding auxiliary verbs when they are separated by adverbs. By using these additional states, the tagging system improved the accuracy from 95.7% to 96.0%. However, the additional context is chosen by analyzing the tagging …

Content truncated to 500 words