Unsupervised Multilingual Learning for POS Tagging

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 1041–1050, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Unsupervised Multilingual Learning for POS Tagging Benjamin Snyder and Tahira Naseem and Jacob Eisenstein and Regina Barzilay Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology 77 Massachusetts Ave., Cambridge MA 02139 {bsnyder, tahira, jacobe, regina}@csail.mit.edu Abstract We demonstrate the effectiveness of multilin- gual learning for unsupervised part-of-speech tagging. The key hypothesis of multilin- gual learning is that by combining cues from multiple languages, the structure of each be- comes more apparent. We formulate a hier- archical Bayesian model for jointly predicting bilingual streams of part-of-speech tags. The model learns language-specific features while capturing cross-lingual patterns in tag distri- bution for aligned words. Once the parame- ters of our model have been learned on bilin- gual parallel data, we evaluate its performance on a held-out monolingual test set. Our evalu- ation on six pairs of languages shows consis- tent and significant performance gains over a state-of-the-art monolingual baseline. For one language pair, we observe a relative reduction in error of 53%. 1 Introduction In this paper, we explore the application of multilin- gual learning to part-of-speech tagging when no an- notation is available. This core task has been studied in an unsupervised monolingual framework for over a decade and is still an active area of research. In this paper, we demonstrate the effectiveness of multilin- gual learning when applied to both closely related and distantly related language pairs. We further ana- lyze the language features which lead to robust bilin- gual performance. The fundamental idea upon which our work is based is that the patterns of ambiguity inherent in part-of-speech tag assignments differ across lan- guages. At the lexical level, a word with part-of- speech tag ambiguity in one language may corre- spond to an unambiguous word in the other lan- guage. For example, the word “can” in English may function as an auxiliary verb, a noun, or a regular verb. However, each of the corresponding functions in Serbian is expressed with a distinct lexical item. Languages also differ in their patterns of structural ambiguity. For example, the presence of an article in English greatly reduces the ambiguity of the suc- ceeding tag. In Serbian, a language without articles, this constraint is obviously absent. The key idea of multilingual learning is that by combining cues from multiple languages, the structure of each becomes more apparent. While multilingual learning can address ambigu- ities in each language, it must be flexible enough to accommodate cross-lingual variations such as tag inventory and syntactic structure. As a result of such variations, two languages often select and order their tags differently even when expressing the same meaning. A key challenge of multilingual learning is to model language-specific structure while allow- ing information to flow between languages. We jointly model bilingual part-of-speech tag se- quences in a hierarchical Bayesian framework. For each word, we posit a hidden tag state which gen- erates the …

Content truncated to 500 words