Developing Machine Translation Engines for Multilingual Participatory Spaces

Comparing and combining tagging with different decoding algorithms for back-translation in NMT: learnings from a low resource scenario Xabier Soto, Olatz Perez-de-Vi˜naspre, Gorka Labaka, Maite Oronoz HiTZ Basque Center for Language Technologies - Ixa, University of the Basque Country UPV/EHU {xabier.soto, olatz.perezdevinaspre, gorka.labaka, maite.oronoz}@ehu.eus Abstract Recently, diverse refinements to the back- translation process have been proposed for improving the performance of Neural Ma- chine Translation (NMT) systems, includ- ing the use of sampling instead of beam search as decoding algorithm, or append- ing a tag to the back-translated corpus. However, not all the combinations of the previous approaches have been tested, re- maining unclear which is the best ap- proach for developing a given NMT sys- tem. In this work, we empirically com- pare and combine existing techniques for back-translation in a real low resource set- ting: the translation of clinical notes from Basque into Spanish. Apart from auto- matically evaluating the NMT systems, we ask bilingual healthcare workers to per- form a human evaluation, and analyze the different synthetic corpora by measuring their lexical diversity. For reproducibil- ity and generalizability, we repeat our ex- periments for German to English transla- tion using public data. The results suggest that in lower resource scenarios tagging only helps when using sampling for decod- ing, complementing the previous literature using bigger corpora from the news do- main. When fine-tuning with a few thou- sand bilingual in-domain sentences, one of our proposed methods (tagged restricted sampling) obtains the best results both in terms of automatic and human evaluation. © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 1 Introduction Neural Machine Translation (NMT) (Kalchbren- ner and Blunsom, 2013; Sutskever et al., 2014) is the state-of-the-art approach for developing Ma- chine Translation (MT) systems. However, as NMT is based on artificial neural networks, its per- formance is dependent on big quantities of bilin- gual sentences, which are not available for all lan- guage pairs and domains. Back-translation (BT) (Sennrich et al., 2016a), based on the automatic translation of a corpus from the target language into the source language for augmenting the training data, has become a de facto standard for improving the performance of NMT models, provided that large monolingual corpora in the target language and domain are available. When generating a translation, considering that looking for all the possible output sentences is practically infeasible, MT systems have to imple- ment an efficient technique for selecting the most probable sentence according to the distribution of the training data. Typically, beam search (Tillmann and Ney, 2003) is used for generating both the out- put sentences of NMT systems and the synthetic sentences produced by BT systems. Edunov et al. (2018) proposed to use sampling for BT as one way to further improve the perfor- mance of NMT systems. Specifically, their ‘un- restricted sampling’1 approach, consisting of ran- domly sampling from the output distribution, ob- tained the best results on average comparing to other decoding algorithms, …

Content truncated to 500 words