Machine Translation for English–Inuktitut with Segmentation, Data Acquisition and Pre-Training

Proceedings of the 5th Conference on Machine Translation (WMT), pages 274–281 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 274 Machine Translation for English–Inuktitut with Segmentation, Data Acquisition and Pre-Training Christian Roest† Lukas Edman‡ Gosse Minnema‡ Kevin Kelly† Jennifer Spenader† Antonio Toral‡ †Institute for Artificial Intelligence ‡Center for Language and Cognition, University of Groningen The Netherlands c.roest@student.rug.nl, j.l.edman@rug.nl, g.f.minnema@rug.nl kevin.kelly@live.se, j.spenader@ai.rug.nl, a.toral.ruiz@rug.nl Abstract Translating to and from low-resource polysyn- thetic languages present numerous challenges for NMT. We present the results of our sys- tems for the English–Inuktitut language pair for the WMT 2020 translation tasks. We in- vestigated the importance of correct morpho- logical segmentation, whether or not adding data from a related language (Greenlandic) helps, and whether using contextual word em- beddings improves translation. While each method showed some promise, the results are mixed. 1 Introduction This paper presents the neural machine transla- tion (NMT) systems submitted by the University of Groningen to the WMT 2020 translation task1 between Inuktitut and English in both directions (EN↔IU), describing both constrained and uncon- strained systems where we investigated the follow- ing research questions: • RQ1. Does morphological segmentation ben- efit translation with polysynthetic languages? Existing NMT research showed that mor- phological segmentation outperforms byte- pair encoding (BPE) (Sennrich et al., 2016) for some agglutinative languages. For ex- ample, rule-based morphological segmen- tation improved English-to-Finnish transla- tion (S´anchez-Cartagena and Toral, 2016). and unsupervised morphological segmen- tation improved Turkish-to-English transla- tion (Ataman et al., 2017). We investigate if morphological segmentation also improves translation performance for polysynthetic lan- guages, and if effects differ depending on translation direction. 1http://www.statmt.org/wmt20/ translation-task.html • RQ2. Does the use of additional data from a related language, Greenlandic (KL), improve the outcome? Due to the scarcity of EN–IU parallel data, we investigate if adding Green- landic data to the Inuktitut data to train a multi- lingual NMT system (Johnson et al., 2017), improves the performance of the NMT sys- tems on the unconstrained task (Zoph et al., 2016). • RQ3. Does the translation benefit from us- ing contextual word embeddings? The use of such embeddings has proven beneficial for many tasks in natural language process- ing (Devlin et al., 2019), including MT (Zhu et al., 2020), so we deem it sensible to test this for a polysynthetic language, which we will do by means of masked language modelling pre-training. In section 2 we present the main data and evalu- ation measures used. In section 3 we present exper- iments with morphological segmentation methods. Section 4 presents the results of our translation sys- tems, and in section 5 we present our conclusions. 2 Corpora and Evaluation The preprocessing followed the procedure of Joanis et al. (2020), carrying out the following steps in or- der: spelling normalisation and romanisation (only for IU), punctuation normalisation, tokenisation, and truecasing (only for EN). Parallel data is addi- tionally filtered (ratio 15, minimum and maximum length 1 and 200, respectively). As monolingual data we use the Common Crawl (CC) corpus for Inuktitut, and the 2019 …

Content truncated to 500 words