Can Synthetic Translations Improve Bitext Quality?

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4753 - 4766 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Can Synthetic Translations Improve Bitext Quality? Eleftheria Briakou and Marine Carpuat Department of Computer Science University of Maryland College Park, MD 20742, USA ebriakou@cs.umd.edu, marine@cs.umd.edu Abstract Synthetic translations have been used for a wide range of NLP tasks primarily as a means of data augmentation. This work explores, in- stead, how synthetic translations can be used to revise potentially imperfect reference trans- lations in mined bitext. We find that syn- thetic samples can improve bitext quality with- out any additional bilingual supervision when they replace the originals based on a semantic equivalence classifier that helps mitigate NMT noise. The improved quality of the revised bi- text is confirmed intrinsically via human evalu- ation and extrinsically through bilingual induc- tion and MT tasks. 1 Introduction While human-written data remains the gold stan- dard to train Neural Machine Translation (NMT) and Multilingual NLP models, there is growing evidence that synthetic bitext samples—sentence- pairs that are translated by NMT—benefit a wide range of tasks. They have been used to enable semi-supervised MT training from monolingual data (Sennrich et al., 2016a; Zhang and Zong, 2016; Hoang et al., 2018), to induce bilingual lexicons (Artetxe et al., 2019; Shi et al., 2021), and to port models trained on one language to an- other (Conneau et al., 2018; Yang et al., 2019). While synthetic bitexts are useful additions to original training data for downstream tasks, it re- mains unclear how they differ from naturally oc- curring data. Some studies suggest that synthetic samples might be simpler and easier to learn (Zhou et al., 2020; Xu et al., 2021). Recognizing that nat- urally occurring bitext can be noisy, for instance, when they are mined from comparable monolin- gual corpora (Resnik and Smith, 2003; Fung and Yee, 1998; Esplà et al., 2019; Schwenk et al., 2021), we hypothesize that synthetic bitext might also di- rectly improve the equivalence of the two bitext sides. Thus synthetic samples might be useful not only for data augmentation but also to revise poten- tially noisy original bitext samples. In this paper, we present a controlled empiri- cal study comparing the quality of bitext mined from monolingual resources with a synthetic ver- sion generated via MT. We focus on the widely used WikiMatrix bitexts for a distant (i.e, EN-EL) and a similar language-pair (i.e, EN-RO), since it has been shown that this corpus contains a signifi- cant proportion of erroneous translations (Caswell et al., 2021). We generate synthetic bitext by trans- lating the original training samples using MT sys- tems trained on the bitext itself and therefore do not inject any additional supervision in the process. We also consider selectively replacing original samples with forward and backward synthetic translations based on a semantic equivalence classifier, which is also trained without additional supervision. We show that the resulting synthetic bitext im- proves the quality of the …

Content truncated to 500 words