IGT2P: From Interlinear Glossed Texts to Paradigms

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5251–5262, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5251 IGT2P: From Interlinear Glossed Texts to Paradigms Sarah Moeller Ling Liu Changbing Yang Katharina Kann Mans Hulden University of Colorado first.last@colorado.edu Abstract An intermediate step in the linguistic analysis of an under-documented language is to find and organize inflected forms that are attested in natural speech. From this data, linguists generate unseen inflected word forms in or- der to test hypotheses about the language’s inflectional patterns and to complete inflec- tional paradigm tables. To get the data lin- guists spend many hours manually creating in- terlinear glossed texts (IGTs). We introduce a new task that speeds this process and automat- ically generates new morphological resources for natural language processing systems: IGT- to-paradigms (IGT2P). IGT2P generates entire morphological paradigms from IGT input. We show that existing morphological reinflection models can solve the task with 21% to 64% ac- curacy, depending on the language. We further find that (i) having a language expert spend only a few hours cleaning the noisy IGT data improves performance by as much as 21 per- centage points, and (ii) POS tags, which are generally considered a necessary part of NLP morphological reinflection input, have no ef- fect on the accuracy of the models considered here. 1 Introduction Over the last few years, multiple shared tasks have encouraged the development of systems for learn- ing morphology, including generating inflected forms of the canonical form—the lemma—of a word. NLP systems that account for morphology can reduce data sparsity caused by an abundance of individual word forms in morphologically rich languages (Cotterell et al., 2016, 2017a, 2018; Mc- Carthy et al., 2019; Vylomova et al., 2020) and help mitigate bias in training data for natural lan- guage processing (NLP) systems (Zmigrod et al., 2019). However, such systems have often been lim- ited to languages with publicly available structured data, i.e. languages for which tables containing Figure 1: Inflected word forms attested in interlin- ear glossed texts (IGT) train transformer encoder- decoder to generalize morphological paradigmatic pat- terns and generate word forms when given known mor- phosyntatic features of missing paradigm cells. Noisy paradigms are automatically constructed from IGT and a language expert creates “cleaned” paradigms. Both sets are tested on the same missing word forms and the results are compared. inflectional patterns can be found, for example, in online dictionaries like Wiktionary.1 This limits the development of NLP systems for morphology to languages for which morphological information can be easily extracted. Here, we propose to instead make use of a re- source which is much more common, especially for low-resource languages: we explore how to leverage interlinear glossed text (IGT)—a com- mon artifact of linguistic field research—to gen- erate unseen forms of inflectional paradigms, as illustrated in Figure 1. This task, which we call IGT-to-paradigms (IGT2P), differs from the ex- 1https://www.wiktionary.org 5252 isting morphological inflection (Yarowsky and Wi- centowski, 2000; Faruqui et al., 2016) task in …

Content truncated to 500 words