Applying Extrasentential Context To Maximum Entropy Based Tagging With A Large Semantic And Syntactic Tagset

Applying Extrasentential Context To Maximum Entropy Based Tagging With A Large Semantic And Syntactic Tagset Ezra Black and Andrew Finch and Ruigiang Zhang ATR Interpreting Telecommunications Laboratories 2-2 Hikaridai Seika-cho, Soraku-gun Kyoto, Japan 619-02 black©it I. atr. co. jp finch©itl, atr. co. jp rzhang©itl, atr. co. jp Abstract Experiments are presented which measure the perplexity reduction derived from incor- porating into the predictive model utilised in a standard tag-n-gram part-of-speech tagger, contextual information from previous sentences of a document. The tagset employed is the roughly-3000-tag ATR General English Tagset, whose tags are both syntactic and semantic in nature. The kind of extrasentential informa- tion provided to the tagger is semantic, and consists in the occurrence or non-occurrence, within the past 6 sentences of the document being tagged, of words tagged with particular tags from the tagset, and of boolean combina- tions of such conditions. In some cases, these conditions are combined with the requirement that the word being tagged belong to a partic- ular set of words thought most likely to ben- efit from the extrasentential information they are being conjoined with. The baseline model utilized is a maximum entropy-based tag-n- gram tagging model, embodying a standard tag-n-gram approach to tagging: i.e. con- straints for tag trigrams, bigrams, and and the word-tag occurrence frequency of the specific word being tagged, form the basis of prediction. Added into to this baseline tagging model is the extrasentential semantic information just indi- cated. The performance of the tagging model with and without the added contextual knowl- edge is contrasted, training from the 850,000- word ATR General English Treebank, and test- ing on the accompanying 53,000-word test tree- bank. Results are that a significant reduction in testset perplexity is achieved via the added se- mantic extrasentential information of the richer model. The model with both long-range tag triggers and more complex linguistic constraints achieved a perplexity reduction of 21.4%. 1 Introduction It appears intuitively that information from earlier sentences in a document ought to help reduce uncertMnty as to a word's correct part- of-speech tag. This is especially so for a large semantic and syntactic tagset such as the roughly-3000-tag ATR General English Tagset (Black et al., 1996; Black et al., 1998). And in fact, (Black et al., 1998) demonstrate a signif- icant "tag trigger-pair" effect. That is, given that certain "triggering" tags have already oc- curred in a document, the probability of oc- currence of specific "triggered" tags is raised significantly--with respect to the unigram tag probability model. Table 1, taken from (Black et al., 1998), provides examples of the tag trigger-pair effect. Yet, it is one thing to show that extrasenten- tial context yields a gain in information with respect to a unigram tag probability model. But it is another thing to demonstrate that extrasentential context supports an improve- ment in perplexity vis-a-vis a part-of-speech tagging model which employs state-of-the-art techniques: such as, for instance, the tag- ging model of a maximum entropy tag-n-gram- based tagger. The present paper undertakes just such …

Content truncated to 500 words