Estimation of Conditional Probabilities With Decision Trees and an Application to Fine-Grained POS Tagging

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 777–784 Manchester, August 2008 Estimation of Conditional Probabilities With Decision Trees and an Application to Fine-Grained POS Tagging Helmut Schmid and Florian Laws IMS, University of Stuttgart {schmid,lawsfn}@ims.uni-stuttgart.de Abstract We present a HMM part-of-speech tag- ging method which is particularly suited for POS tagsets with a large number of fine-grained tags. It is based on three ideas: (1) splitting of the POS tags into attribute vectors and decomposition of the contex- tual POS probabilities of the HMM into a product of attribute probabilities, (2) esti- mation of the contextual probabilities with decision trees, and (3) use of high-order HMMs. In experiments on German and Czech data, our tagger outperformed state- of-the-art POS taggers. 1 Introduction A Hidden-Markov-Model part-of-speech tagger (Brants, 2000, e.g.) computes the most probable POS tag sequence ˆtN 1 = ˆt1, ..., ˆtN for a given word sequence wN 1 . ˆtN 1 = arg max tN 1 p(tN 1 , wN 1 ) The joint probability of the two sequences is de- fined as the product of context probabilities and lexical probabilities over all POS tags: p(tN 1 , wN 1 ) = N � i=1 p(ti|ti−1 i−k) � �� � context prob. �p(wi|ti) �� � lexical prob. (1) HMM taggers are fast and were successfully ap- plied to a wide range of languages and training cor- pora. c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. POS taggers are usually trained on corpora with between 50 and 150 different POS tags. Tagsets of this size contain little or no information about number, gender, case and similar morphosyntac- tic features. For languages with a rich morphol- ogy such as German or Czech, more fine-grained tagsets are often considered more appropriate. The additional information may also help to disam- biguate the (base) part of speech. Without gender information, for instance, it is difficult for a tagger to correctly disambiguate the German sentence Ist das Realit¨at? (Is that reality?). The word das is ambiguous between an article and a demonstrative. Because of the lack of gender agreement between das (neuter) and the noun Realit¨at (feminine), the article reading must be wrong. The German Tiger treebank (Brants et al., 2002) is an example of a corpus with a more fine-grained tagset (over 700 tags overall). Large tagsets aggra- vate sparse data problems. As an example, take the German sentence Das zu versteuernde Einkommen sinkt (“The to be taxed income decreases”; The taxable income decreases). This sentence should be tagged as shown in table 1. Das ART.Def.Nom.Sg.Neut zu PART.Zu versteuernde ADJA.Pos.Nom.Sg.Neut Einkommen N.Reg.Nom.Sg.Neut sinkt VFIN.Full.3.Sg.Pres.Ind . SYM.Pun.Sent Table 1: Correct POS tags for the German sentence Das zu versteuernde Einkommen sinkt. Unfortunately, the POS trigram consisting of the tags of the first three words does not occur in the Tiger corpus. (Neither does the pair con- sisting of the first two tags.) The unsmoothed 777 context probability of the third POS …

Content truncated to 500 words