Pushing the Limits of Low-Resource Morphological Inflection

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 984–996, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 984 Pushing the Limits of Low-Resource Morphological Inflection Antonios Anastasopoulos and Graham Neubig Language Technologies Institute, Carnegie Mellon University {aanastas,gneubig}@cs.cmu.edu Abstract Recent years have seen exceptional strides in the task of automatic morphological inflec- tion generation. However, for a long tail of languages the necessary resources are hard to come by, and state-of-the-art neural methods that work well under higher resource settings perform poorly in the face of a paucity of data. In response, we propose a battery of im- provements that greatly improve performance under such low-resource conditions. First, we present a novel two-step attention architecture for the inflection decoder. In addition, we in- vestigate the effects of cross-lingual transfer from single and multiple languages, as well as monolingual data hallucination. The macro- averaged accuracy of our models outperforms the state-of-the-art by 15 percentage points.1 Also, we identify the crucial factors for suc- cess with cross-lingual transfer for morpho- logical inflection: typological similarity and a common representation across languages. 1 Introduction The majority of the world’s languages are cate- gorized as synthetic, meaning that they have rich morphology, be it fusional, agglutinative, polysyn- thetic, or a mixture thereof. As Natural Language Processing (NLP) keeps expanding its frontiers to encompass more and more languages, modeling of the grammatical functions that guide language generation is of utmost importance. In the case of morphologically-rich languages, explicit modeling of the inflection processes has significant potential to alleviate issues created by data scarcity and the resulting lack of vocabu- lary coverage. Especially on low-resource, under- represented languages and dialects, the potential for impact is much higher. For example, speech 1Our code is available at https://github.com/ antonisa/inflection. a g u à t1 · · · tM encoder ht 1 · · · ht M ct 1 · · · ct K attention s1 · · · sK x1 · · · xN encoder hx 1 · · · hx N cx 1 · · · cx K attention s′ 1 · · · s′ K softmax P(y1 · · · yK) decoder V PRS 2 PL IND a g u a r Figure 1: Visualization of our proposed two-step atten- tion architecture. The decoder first attends over the tag sequence T and then uses the updated decoder state s′ to attend over the character sequence X in order to pro- duce the inflected form Y. (Example from Asturian.) recognition (Foley et al., 2018) and predictive key- boards (Breiner et al., 2019) for under-represented languages, if they exist, largely still rely on uni- gram lexicons with performance inferior to the so- phisticated language models used in high-resource ones. Good inflection models would be invaluable for predictive text technology in morphologically- rich languages as they could effectively enable proper handling of the huge vocabulary. Additionally, they could be very useful for building …

Content truncated to 500 words