Improved Unsupervised POS Induction through Prototype Discovery

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1298–1307, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Improved Unsupervised POS Induction through Prototype Discovery Omri Abend1∗ Roi Reichart2 Ari Rappoport1 1Institute of Computer Science, 2ICNC Hebrew University of Jerusalem {omria01|roiri|arir}@cs.huji.ac.il Abstract We present a novel fully unsupervised al- gorithm for POS induction from plain text, motivated by the cognitive notion of proto- types. The algorithm first identifies land- mark clusters of words, serving as the cores of the induced POS categories. The rest of the words are subsequently mapped to these clusters. We utilize morpho- logical and distributional representations computed in a fully unsupervised manner. We evaluate our algorithm on English and German, achieving the best reported re- sults for this task. 1 Introduction Part-of-speech (POS) tagging is a fundamental NLP task, used by a wide variety of applications. However, there is no single standard POS tag- ging scheme, even for English. Schemes vary significantly across corpora and even more so across languages, creating difficulties in using POS tags across domains and for multi-lingual systems (Jiang et al., 2009). Automatic induction of POS tags from plain text can greatly alleviate this problem, as well as eliminate the efforts in- curred by manual annotations. It is also a problem of great theoretical interest. Consequently, POS induction is a vibrant research area (see Section 2). In this paper we present an algorithm based on the theory of prototypes (Taylor, 2003), which posits that some members in cognitive categories are more central than others. These practically de- fine the category, while the membership of other elements is based on their association with the ∗ Omri Abend is grateful to the Azrieli Foundation for the award of an Azrieli Fellowship. central members. Our algorithm first clusters words based on a fine morphological representa- tion. It then clusters the most frequent words, defining landmark clusters which constitute the cores of the categories. Finally, it maps the rest of the words to these categories. The last two stages utilize a distributional representation that has been shown to be effective for unsupervised parsing (Seginer, 2007). We evaluated the algorithm in both English and German, using four different mapping-based and information theoretic clustering evaluation mea- sures. The results obtained are generally better than all existing POS induction algorithms. Section 2 reviews related work. Sections 3 and 4 detail the algorithm. Sections 5, 6 and 7 describe the evaluation, experimental setup and results. 2 Related Work Unsupervised and semi-supervised POS tagging have been tackled using a variety of methods. Sch¨utze (1995) applied latent semantic analysis. The best reported results (when taking into ac- count all evaluation measures, see Section 5) are given by (Clark, 2003), which combines dis- tributional and morphological information with the likelihood function of the Brown algorithm (Brown et al., 1992). Clark’s tagger is very sen- sitive to its initialization. Reichart et al. (2010b) propose a method to identify the high quality runs of this algorithm. In this paper, …

Content truncated to 500 words