Data Augmentation with Diversified Rephrasing for Low-Resource Neural Machine Translation

Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, pages 35–47 September 4–8, 2023, Macau SAR, China. ©2023 The authors. This article is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). 35 Data Augmentation with Diversified Rephrasing for Low-Resource Neural Machine Translation Yuan Gao y.gao1@massey.ac.nz Feng Hou∗ f.hou@massey.ac.nz Huia Jahnke h.t.jahnke@massey.ac.nz Ruili Wang ruili.wang@massey.ac.nz SMCS, Massey University, Auckland, 0632, New Zealand Abstract Data augmentation is an effective way to enhance the performance of neural machine transla- tion models, especially for low-resource languages. Existing data augmentation methods are either at a token level or a sentence level. The data augmented using token level methods lack syntactic diversity and may alter original meanings. Sentence level methods usually generate low-quality source sentences that are not semantically paired with the original target sentences. In this paper, we propose a novel data augmentation method to generate diverse, high-quality and meaning-preserved new instances. Our method leverages high-quality translation models trained with high-resource languages to rephrase an original sentence by translating it into an intermediate language and then back to the original language. Through this process, the high- performing translation models guarantee the quality of the rephrased sentences, and the syn- tactic knowledge from the intermediate language can bring syntactic diversity to the rephrased sentences. Experimental results show our method can enhance the performance in various low- resource machine translation tasks. Moreover, by combining our method with other techniques that facilitate NMT, we can yield even better results. 1 Introduction Current neural machine translation (NMT) (Ng et al., 2019; Wang et al., 2021; Wei et al., 2022; Shao and Feng, 2022) systems, especially those based on Transformer (Vaswani et al., 2017), have achieved human-level performance in translation quality (Hassan et al., 2018; Popel et al., 2020). These systems are trained using hundreds of millions of sentence pairs to ensure that they can generalize to unseen instances. However, large-scale parallel data is scarce and only available for a few high-resource language pairs (Lample et al., 2018; Haddow et al., 2022). Thus, the generalization of low-resource NMT models is far below an acceptable standard. Recently, data augmentation (Sennrich et al., 2016a; Gao et al., 2019; Provilkov et al., 2020; Nguyen et al., 2020; Wei et al., 2022) has shown to be an effective way to improve the generalization of NMT models, especially for low-resource languages (Currey et al., 2017). Existing data augmentation methods for NMT can be categorized into token level or sentence level methods. Token level methods randomly replace words with rare words in both source and target sides to enhance the translation of rare words (Fadaee et al., 2017), or introduce *Corresponding author 36 token level noises in the source side (Sennrich et al., 2016a; Lample et al., 2018; Artetxe et al., 2018; Wang et al., 2018; Gao et al., 2019; Provilkov et al., 2020) to improve the robustness of models (Khayrallah and Koehn, 2018). Sentence level methods are mainly based on back- translation (Sennrich et al., 2016b; Edunov et al., 2018), …

Content truncated to 500 words