Mistake-Driven Mixture of Hierarchical Tag Context Trees

Mistake-Driven Mixture of Hierarchical Tag Context Trees Masahiko Haruno NTT Communication Science Laboratories 1-1 Hikari-No-Oka Yokosuka-Shi Kanagawa 239, Japan haruno©cslab, kecl. ntt. co. j p Yuji Matsumoto NAIST 8916-5 Takayama-cho Ikoma-Shi Nara 630-01, Japan mat su©is, aist-nara, ac. j p Abstract This paper proposes a mistake-driven mix- ture method for learning a tag model. The method iteratively performs two proce- dures: 1. constructing a tag model based on the current data distribution and 2. updating the distribution by focusing on data that are not well predicted by the constructed model. The final tag model is constructed by mixing all the models according to their performance. To well reflect the data distribution, we repre- sent each tag model as a hierarchical tag (i.e.,NTT 1 < proper noun < noun) con- text tree. By using the hierarchical tag context tree, the constituents of sequential tag models gradually change from broad coverage tags (e.g.,noun) to specific excep- tional words that cannot be captured by generM tags. In other words, the method incorporates not only frequent connec- tions but also infrequent ones that are of- ten considered to be collocationah We evaluate several tag models by implement- ing Japanese part-of-speech taggers that share all other conditions (i.e.,dictionary and word model) other than their tag models. The experimental results show the proposed method significantly outper- forms both hand-crafted and conventional statistical methods. 1 Introduction The last few years have seen the great success of stochastic part-of-speech (POS) taggers (Church, 1988: Kupiec, 1992; Charniak et M., 1993; Brill, 1992; Nagata, 1994). The stochastic approach gen- erally attains 94 to 96% accuracy and replaces the labor-intensive compilation of linguistics rules by using an automated learning algorithm. However, 1NTT is an abbreviation of Nippon Telegraph and Telephone Corporation. practical systems require more accuracy because POS tagging is an inevitable pre-processing step for all practical systems. To derive a new stochastic tagger, we have two options since stochastic taggers generally comprise two components: word model and tag model. The word model is a set of probabilities that a word oc- curs with a tag (part-of-speech) when given the pre- ceding words and their tags in a sentence. On the contrary, the tag model is a set of probabilities that a tag appears after the preceding words and their tags. The first option is to construct more sophisticated word models. (Charniak et al., 1993) reports that their model considers the roots and suffixes of words to greatly improve tagging accuracy for English cor- pora. However, the word model approach has the following shortcomings: • For agglutinative languages such as Japanese and Chinese, the simple Bayes transfer rule is inapplicable because the word length of a sen- tence is not fixed in all possible segmentations -~. We can only use simpler word models in these languages. • Sophisticated word models largely depend on the target language. It is time-consuming to compile fine-grained word models for each lan- guage. The second option is to devise a new tag model. (Sch~tze and Singer. …

Content truncated to 500 words