EM Can Find Pretty Good HMM POS-Taggers (When Given a Good Start)

Proceedings of ACL-08: HLT, pages 746–754, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics EM Can Find Pretty Good HMM POS-Taggers (When Given a Good Start)∗ Yoav Goldberg and Meni Adler and Michael Elhadad Ben Gurion University of the Negev Department of Computer Science POB 653 Be’er Sheva, 84105, Israel {yoavg,adlerm,elhadad}@cs.bgu.ac.il Abstract We address the task of unsupervised POS tag- ging. We demonstrate that good results can be obtained using the robust EM-HMM learner when provided with good initial conditions, even with incomplete dictionaries. We present a family of algorithms to compute effective initial estimations p(t|w). We test the method on the task of full morphological disambigua- tion in Hebrew achieving an error reduction of 25% over a strong uniform distribution base- line. We also test the same method on the stan- dard WSJ unsupervised POS tagging task and obtain results competitive with recent state-of- the-art methods, while using simple and effi- cient learning methods. 1 Introduction The task of unsupervised (or semi-supervised) part- of-speech (POS) tagging is the following: given a dictionary mapping words in a language to their pos- sible POS, and large quantities of unlabeled text data, learn to predict the correct part of speech for a given word in context. The only supervision given to the learning process is the dictionary, which in a realistic scenario, contains only part of the word types observed in the corpus to be tagged. Unsupervised POS tagging has been traditionally approached with relative success (Merialdo, 1994; Kupiec, 1992) by HMM-based generative mod- els, employing EM parameters estimation using the Baum-Welch algorithm. However, as recently noted ∗This work is supported in part by the Lynn and William Frankel Center for Computer Science. by Banko and Moore (2004), these works made use of filtered dictionaries: dictionaries in which only relatively probable analyses of a given word are pre- served. This kind of filtering requires serious su- pervision: in theory, an expert is needed to go over the dictionary elements and filter out unlikely anal- yses. In practice, counts from an annotated corpus have been traditionally used to perform the filtering. Furthermore, these methods require rather compre- hensive dictionaries in order to perform well. In recent work, researchers try to address these deficiencies by using dictionaries with unfiltered POS-tags, and testing the methods on “diluted dic- tionaries” – in which many of the lexical entries are missing (Smith and Eisner, 2005) (SE), (Goldwater and Griffiths, 2007) (GG), (Toutanova and Johnson, 2008) (TJ). All the work mentioned above focuses on unsu- pervised English POS tagging. The dictionaries are all derived from tagged English corpora (all recent work uses the WSJ corpus). As such, the setting of the research is artificial: there is no reason to per- form unsupervised learning when an annotated cor- pus is available. The problem is rather approached as a workbench for exploring new learning methods. The result is a series of creative algorithms, that have steadily improved results on the same dataset: unsu- pervised CRF training using contrastive …

Content truncated to 500 words