One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1993–2003 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1182 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1993–2003 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1182 One-Shot Neural Cross-Lingual Transfer for Paradigm Completion Katharina Kann CIS LMU Munich, Germany kann@cis.lmu.de Ryan Cotterell Department of Computer Science Johns Hopkins University, USA ryan.cotterell@jhu.edu Hinrich Sch¨utze CIS LMU Munich, Germany inquiries@cislmu.org Abstract We present a novel cross-lingual transfer method for paradigm completion, the task of mapping a lemma to its inflected forms, using a neural encoder-decoder model, the state of the art for the monolingual task. We use labeled data from a high-resource language to increase performance on a low- resource language. In experiments on 21 language pairs from four different language families, we obtain up to 58% higher ac- curacy than without transfer and show that even zero-shot and one-shot learning are possible. We further find that the degree of language relatedness strongly influences the ability to transfer morphological knowl- edge. 1 Introduction Low-resource natural language processing (NLP) remains an open problem for many tasks of interest. Furthermore, for most languages in the world, high- cost linguistic annotation and resource creation are unlikely to be undertaken in the near future. In the case of morphology, out of the 7000 currently spo- ken (Lewis, 2009) languages, only about 200 have computer-readable annotations (Sylak-Glassman et al., 2015) – although morphology is easy to an- notate compared to syntax and semantics. Transfer learning is one solution to this problem: it exploits annotations in a high-resource language to train a system for a low-resource language. In this work, we present a method for cross-lingual transfer of inflectional morphology using an encoder-decoder recurrent neural network (RNN). This allows for the development of tools for computational mor- phology with limited annotated data. In many languages, individual lexical entries may be realized as distinct inflections of a single Present Past Indicative Indicative Sg Pl Sg Pl 1 sue˜no so˜namos so˜n´e so˜namos 2 sue˜nas so˜n´ais so˜naste so˜nasteis 3 sue˜na sue˜nan so˜n´o so˜naron Table 1: Partial inflection table for the Spanish verb so˜nar. lemma depending on the syntactic context. For ex- ample, the 3SgPresInd of the English verbal lemma to bring is brings. In morphologically rich lan- guages, a lemma can have hundreds of individ- ual forms. Thus, both generation and analysis of such morphological inflections are active areas of research in NLP and morphological process- ing has been shown to be a boon to several other down-stream applications, e.g., machine transla- tion (Dyer et al., 2008), speech recognition (Creutz et al., 2007), parsing (Seeker and C¸ etino˘glu, 2015), keyword spotting (Narasimhan et al., 2014) and word embeddings (Cotterell et al., 2016b), inter alia. In this work, we focus on paradigm comple- tion, a form of morphological generation that maps a given lemma to a target inflection, e.g., (bring, Past) �→ …

Content truncated to 500 words