Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 600–609, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections Dipanjan Das∗ Carnegie Mellon University Pittsburgh, PA 15213, USA dipanjan@cs.cmu.edu Slav Petrov Google Research New York, NY 10011, USA slav@google.com Abstract We describe a novel approach for inducing unsupervised part-of-speech taggers for lan- guages that have no labeled training data, but have translated text in a resource-rich lan- guage. Our method does not assume any knowledge about the target language (in par- ticular no tagging dictionary is assumed), making it applicable to a wide array of resource-poor languages. We use graph-based label propagation for cross-lingual knowl- edge transfer and use the projected labels as features in an unsupervised model (Berg- Kirkpatrick et al., 2010). Across eight Eu- ropean languages, our approach results in an average absolute improvement of 10.4% over a state-of-the-art baseline, and 16.7% over vanilla hidden Markov models induced with the Expectation Maximization algorithm. 1 Introduction Supervised learning approaches have advanced the state-of-the-art on a variety of tasks in natural lan- guage processing, resulting in highly accurate sys- tems. Supervised part-of-speech (POS) taggers, for example, approach the level of inter-annotator agreement (Shen et al., 2007, 97.3% accuracy for English). However, supervised methods rely on la- beled training data, which is time-consuming and expensive to generate. Unsupervised learning ap- proaches appear to be a natural solution to this prob- lem, as they require only unannotated text for train- ∗This research was carried out during an internship at Google Research. ing models. Unfortunately, the best completely un- supervised English POS tagger (that does not make use of a tagging dictionary) reaches only 76.1% ac- curacy (Christodoulopoulos et al., 2010), making its practical usability questionable at best. To bridge this gap, we consider a practically mo- tivated scenario, in which we want to leverage ex- isting resources from a resource-rich language (like English) when building tools for resource-poor for- eign languages.1 We assume that absolutely no la- beled training data is available for the foreign lan- guage of interest, but that we have access to parallel data with a resource-rich language. This scenario is applicable to a large set of languages and has been considered by a number of authors in the past (Al- shawi et al., 2000; Xi and Hwa, 2005; Ganchev et al., 2009). Naseem et al. (2009) and Snyder et al. (2009) study related but different multilingual gram- mar and tagger induction tasks, where it is assumed that no labeled data at all is available. Our work is closest to that of Yarowsky and Ngai (2001), but differs in two important ways. First, we use a novel graph-based framework for project- ing syntactic information across language bound- aries. To this end, we construct a bilingual graph over word types to establish a connection between the two languages (§3), and then use graph label propagation to project syntactic information from English to the foreign language (§4). Second, we …

Content truncated to 500 words