Unsupervised Cross-Lingual Part-of-Speech Tagging for Truly Low-Resource Scenarios

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 4820–4831, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 4820 Unsupervised Cross-Lingual Part-of-Speech Tagging for Truly Low-Resource Scenarios Ramy Eskander Department of Computer Science Columbia University rnd2110@columbia.edu Smaranda Muresan Data Science Institute Columbia University smara@columbia.edu Michael Collins Department of Computer Science Columbia University mc3354@columbia.edu Abstract We describe a fully unsupervised cross-lingual transfer approach for part-of-speech (POS) tagging under a truly low resource scenario. We assume access to parallel translations be- tween the target language and one or more source languages for which POS taggers are available. We use the Bible as parallel data in our experiments: small size, out-of-domain and covering many diverse languages. Our approach innovates in three ways: 1) a ro- bust approach of selecting training instances via cross-lingual annotation projection that ex- ploits best practices of unsupervised type and token constraints, word-alignment confidence and density of projected POS, 2) a Bi-LSTM architecture that uses contextualized word em- beddings, affix embeddings and hierarchical Brown clusters, and 3) an evaluation on 12 di- verse languages in terms of language family and morphological typology. In spite of the use of limited and out-of-domain parallel data, our experiments demonstrate significant im- provements in accuracy over previous work. In addition, we show that using multi-source in- formation, either via projection or output com- bination, improves the performance for most target languages. 1 Introduction Majority of world’s languages do not have anno- tated datasets even for the most simple NLP tasks such as part-of-speech (POS) tagging. However, ef- forts in documenting low-resource languages often contain translations, usually of religious text, into other high-resource languages. One such parallel corpus is the Bible (Mayer and Cysouw, 2014): 484 languages have a complete Bible translation, while 2551 have a part of the Bible translated. Our goal is to learn POS taggers for a diverse set of target languages in a truly low-resource scenario, where only a limited and possibly out-of-domain set of translations into one or more high-resource languages is available (e.g., the Bible), together with supervised POS taggers for the high-resource source language(s). Unsupervised cross-lingual POS tagging via an- notation projection has a long research history (Yarowsky et al., 2001; Fossum and Abney, 2005; Das and Petrov, 2011; Duong et al., 2013; Agi´c et al., 2015, 2016; Buys and Botha, 2016). In con- trast to our work, these approaches either use large and/or in-domain parallel data or rely on a large number of source languages for projection. How- ever, since projection could suffer from bad transla- tion, alignment mistakes or wrong assumptions, a key consideration for all these approaches is how to obtain high-quality training instances for the target language (i.e., sentences with accurate POS tags projected from the source-language(s)). Coupling token and type constraints (Das and Petrov, 2011; T¨ackstr¨om et al., 2013; Buys and Botha, 2016), word-alignment confidence (Duong et al., 2013), multi-source projection (Agi´c et al., 2016) and coverage (percentage of tokens covered by multi- source projection) (Plank and Agi´c, …

Content truncated to 500 words