Getting More Data for Low-resource Morphological Inflection: Language Models and Data Augmentation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3978–3983 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3978 Getting More Data for Low-resource Morphological Inflection: Language Models and Data Augmentation Alexey Sorokin Moscow State University, Moscow Institute of Physics and Technology Moscow, Leninskie Gory, GSP-1, Faculty of Mathematics and Mechanics, name dot surname at list dot ru Abstract We investigate the effect of data augmentation on low-resource morphological segmentation. We compare two settings: the pure low-resource one, when only 100 annotated word forms are available, and the augmented one, where we use the original training set and 1000 unlabeled word forms to generate 1000 artificial inflected forms. Evaluating on Sigmorphon 2018 dataset, we observe that using the best among these two models reduces the error rate of state-of-the-art model by 6%, while for our baseline model the error reduction is 17%. Keywords: inflection, encoder-decoder, abstract paradigms, language models, data augmentation 1. Introduction Morphological inflection is the process that generates the word form given its lexeme and morphological properties. For example, the inputs volver and V;PRS;1;SG produce the inflected form vuelvo. This task can be an intermedi- ate stage of text generation, especially in pattern-based ap- proaches. It is also significant by itself for creation and expansion of lexical and morphological resources. Mor- phological inflection is especially important and challeng- ing for low-resource languages, where no or little annotated data is present. As any string-to-string task, morphological inflection can be solved by attention-based encoder-decoder architec- tures(Kann and Sch¨utze, 2016). The effectiveness of differ- ent approaches was thoroughly tested during multiple edi- tions of Sigmorphon Shared Task on Morphological Inflec- tion.1 However, its 2018 edition demonstrated that in high- resource conditions (10000 inflection examples for train- ing) several systems are almost perfect, achieving average accuracy above 96% across 100 languages, whereas for a significant part of these languages the top accuracy exceeds 99%(Cotterell et al., 2018). For medium (1000 word forms) and low (100 word forms) settings the results are also sat- isfiable, however, a plenty room for improvement remains. Especially hard is the low setting, when some morpholog- ical tags may have no training forms. Indeed, in some lan- guages (e.g., Basque) 100 words cannot cover even a sin- gle verb paradigm. However, even in such restricted con- ditions some systems perform significantly better than oth- ers, the state-of-the-art approach is imitation learning via minimization of Levenshtein distance between the network output and the correct word form (Makarov and Clematide, 2018b). This model is built on top of the previous systems (Aharoni and Goldberg, 2017; Makarov et al., 2017) that use monotonic attention to generate a sequence of string edits. We examine two ways of providing more data to the model. The first are language models. There are multiple ways to adapt them to this task: language models can be used to ad- ditionally ensure that the generated string satisfies the rules 1sigmorphon.github.io of the language, e.g. phonetic. This approach is …

Content truncated to 500 words