A Simple Unsupervised Learner for POS Disambiguation Rules Given Only a Minimal Lexicon

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 688–697, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP A Simple Unsupervised Learner for POS Disambiguation Rules Given Only a Minimal Lexicon Qiuye Zhao Mitch Marcus Dept. of Computer & Information Science University of Pennsylvania qiuye, mitch@cis.upenn.edu Abstract We propose a new model for unsupervised POS tagging based on linguistic distinc- tions between open and closed-class items. Exploiting notions from current linguis- tic theory, the system uses far less infor- mation than previous systems, far simpler computational methods, and far sparser descriptions in learning contexts. By ap- plying simple language acquisition tech- niques based on counting, the system is given the closed-class lexicon, acquires a large open-class lexicon and then acquires disambiguation rules for both. This sys- tem achieves a 20% error reduction for POS tagging over state-of-the-art unsuper- vised systems tested under the same con- ditions, and achieves comparable accuracy when trained with much less prior infor- mation. 1 Introduction All recent research on unsupervised tagging, as well as the majority of work on supervised tag- gers, views POS tagging as a sequential labeling problem and treats all POS tags, both closed- and open-class, as roughly equivalent. In this work we explore a different understanding of the tagging problem, viewing it as a process of first identifying functional syntactic contexts, which are flagged by closed-class items, and then using these func- tional contexts to determine the POS labels. This disambiguation model differs from most previous work in three ways: 1) it uses different encod- ings over two distinct domains (roughly open- and closed-class words) with complementary distribu- tion (and so decodes separately); 2) it is determin- istic and 3) it is non-lexicalized. By learning dis- ambiguation models for open- and closed- classes separately, we found that the deterministic, rule- based model can be learned from unannotated data by a simple strategy of selecting a rule in each ap- propriate context with the highest count. In contrast to this, most previous work on un- supervised tagging (especially for English) con- centrates on improving the parameter estima- tion techniques for training statistical disambigua- tion models from unannotated data. For exam- ple, (Smith&Eisner, 2005) proposes contrastive estimation (CE) for log-linear models (CRF), achieving the current state-of-the-art performance of 90.4%; (Goldwater&Griffiths, 2007) applies a Bayesian approach to improve maximum- likelihood estimation (MLE) for training genera- tive models (HMM). In the main experiments of both of these papers, the disambiguation model is learned, but the algorithms assume a complete knowledge of the lexicon with all possible tags for each word. In this work, we propose making such a large lexicon unnecessary by learning the bulk of the lexicon along with learning a disambigua- tion model. Little previous work has been done on this nat- ural and simple idea because the clusters found by previous induction schemes are not in line with the lexical categories that we care about. (Chan, 2008) is perhaps the first with the intention of generat- ing ”a …

Content truncated to 500 words