Part-of-speech Taggers for Low-resource Languages using CCA Features

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1292–1302, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Part-of-speech Taggers for Low-resource Languages using CCA Features Young-Bum Kim† Benjamin Snyder‡ Ruhi Sarikaya† †Microsoft Corporation, Redmond, WA ‡University of Wisconsin-Madison, Madison, WI {ybkim, ruhi.sarikaya}@microsoft.com bsnyder@cs.wisc.edu Abstract In this paper, we address the challenge of creating accurate and robust part- of-speech taggers for low-resource lan- guages. We propose a method that lever- ages existing parallel data between the tar- get language and a large set of resource- rich languages without ancillary resources such as tag dictionaries. Crucially, we use CCA to induce latent word represen- tations that incorporate cross-genre distri- butional cues, as well as projected tags from a full array of resource-rich lan- guages. We develop a probability-based confidence model to identify words with highly likely tag projections and use these words to train a multi-class SVM using the CCA features. Our method yields average performance of 85% accuracy for languages with almost no resources, outperforming a state-of-the-art partially- observed CRF model. 1 Introduction We address the challenge of creating accurate and robust part-of-speech taggers for low-resource languages. We aim to apply our methods to the hundreds, and potentially thousands, of languages with meager electronic resources. We do not as- sume the existence of a tag dictionary, or any other sort of prior knowledge of the target language. In- stead, we base our methods entirely on the exis- tence of parallel data between the target language and a set of resource-rich languages. Fortunately, such parallel data exists for just about every written language, in the form of Bible translations. Around 2,500 languages have at least partial Bible translations, and somewhere between 500 and 1,000 languages have complete transla- tions. We have collected such electronic Bible translations for 650 languages. Figure 1 breaks down the number of languages in our collection according to their token count. The majority of our languages have at least 200,000 tokens of Bible translations. While previous studies (T¨ackstr¨om et al., 2013; Ganchev and Das, 2013) have addressed this gen- eral setting, they have typically assumed the exis- tence of a partial tag dictionary as well as large quantities of non-parallel data in the target lan- guage. These assumptions are quite reasonable for the dozen most popular languages in the world, but are inadequate for the creation of a truly world- wide repository of NLP tools and linguistic data. In fact, we argue that such ancillary sources of information are not really necessary once we take into account the vastly multilingual nature of our parallel data. Annotations projected from individ- ual resource-rich languages are often noisy and unreliable, due to systematic differences between the languages in question, as well as word align- ment errors. We can thus think of these languages as very lazy and unreliable annotators of our tar- get language. Despite their incompetence, as the number of such annotators increases, their com- bined efforts converge upon the truth, as idiosyn- …

Content truncated to 500 words