Translation Memories as Baselines for Low-Resource Machine Translation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6759–6767 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6759 Translation Memories as Baselines for Low-Resource Machine Translation Rebecca Knowles, Patrick Littell National Research Council Canada 1200 Montreal Road, Ottawa ON, K1A 0R6 {Rebecca.Knowles, Patrick.Littell}@nrc-ncrc.gc.ca Abstract Low-resource machine translation research often requires building baselines to benchmark estimates of progress in translation quality. Neural and statistical phrase-based systems are often used with out-of-the-box settings to build these initial baselines before analyzing more sophisticated approaches, implicitly comparing the first machine translation system to the absence of any translation assistance. We argue that this approach overlooks a basic resource: if you have parallel text, you have a translation memory. In this work, we show that using available text as a translation memory baseline against which to compare machine translation systems is simple, effective, and can shed light on additional translation challenges. Keywords: machine translation, low-resource, translation memories 1. Introduction The goal of machine translation (MT) is to take as input heretofore unobserved sequences of text in the source language and produce as output accurate and fluent translations of that same text in the target lan- guage. When it comes to low-resource language pairs, this is an especially challenging task. When building supervised1 machine translation systems or introducing a new dataset for a low-resource language pair, the typ- ical approach is to build baseline systems using well- known neural or phrase based statistical machine trans- lation architectures using all available parallel text in the languages (perhaps aside from a held-out test and/or validation set). For some language pairs, this could be on the order of just thousands of sentence pairs. While the field continues to progress, automatic and human evaluations of many low-resource machine translation systems demonstrate that they are still far from reach- ing the point where they can be consistently useful for machine translation’s downstream applications: com- prehension, communication, or publication. Very low-resource machine translation is often of too low quality to use directly, without any sort of interven- tion or improvement. Consequently, researchers might hope that an initial machine translation system could be useful to human translators as part of a computer aided translation (CAT) pipeline (i.e., post-editing, interac- tive translation, etc.). When we build a first machine translation baseline, that first baseline is implicitly compared against having no translation assistance at all. However, many translators working in CAT work- flows do have access to translation tools: dictionaries, spell checkers, and translation memories. A transla- tion memory (TM) is a collection of source-target seg- 1In this work, we focus only on supervised machine trans- lation, not the scenario where there is no parallel text avail- able. ment pairs which are (human-produced and/or human- validated) translations of one another. When preparing to translate a novel sentence, the CAT tool can present translations of similar sentences from the TM to the translator, which they can then modify. If you have parallel text suitable for machine …

Content truncated to 500 words