AugVic: Exploiting BiText Vicinity for Low-Resource NMT

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3034–3045 August 1–6, 2021. ©2021 Association for Computational Linguistics 3034 AUGVIC: Exploiting BiText Vicinity for Low-Resource NMT Tasnim Mohiuddin∗ ¶, M Saiful Bari ∗¶, and Shafiq Joty¶† ¶Nanyang Technological University, Singapore †Salesforce Research Asia, Singapore {mohi0004, bari0001, srjoty}@ntu.edu.sg Abstract The success of Neural Machine Translation (NMT) largely depends on the availability of large bitext training corpora. Due to the lack of such large corpora in low-resource language pairs, NMT systems often exhibit poor perfor- mance. Extra relevant monolingual data of- ten helps, but acquiring it could be quite ex- pensive, especially for low-resource languages. Moreover, domain mismatch between bitext (train/test) and monolingual data might de- grade the performance. To alleviate such is- sues, we propose AUGVIC, a novel data aug- mentation framework for low-resource NMT which exploits the vicinal samples of the given bitext without using any extra monolingual data explicitly. It can diversify the in-domain bitext data with finer level control. Through extensive experiments on four low-resource language pairs comprising data from different domains, we have shown that our method is comparable to the traditional back-translation that uses extra in-domain monolingual data. When we combine the synthetic parallel data generated from AUGVIC with the ones from the extra monolingual data, we achieve further improvements. We show that AUGVIC helps to attenuate the discrepancies between relevant and distant-domain monolingual data in tradi- tional back-translation. To understand the con- tributions of different components of AUGVIC, we perform an in-depth framework analysis. 1 Introduction Neural Machine Transaltion (NMT) has shown impressive performance in high-resource settings, even claiming to achieve parity with human profes- sional translators (Hassan et al., 2018; Popel et al., 2020). Most successful NMT systems have billions of parameters (Lepikhin et al., 2021). They gener- ally work well only when a good amount of par- allel training data is available and perform poorly ∗Equal contribution in low-resource conditions (Koehn and Knowles, 2017; Guzm´an et al., 2019). However, majority of the languages are low-resourced despite being used by large portion of world population. Hence, improving low-resource MT quality has been of great interests to the MT researchers. There have been several attempts to extend the success of NMT in high-resource settings to low- resource language pairs that have a relatively small amount of available parallel data. Most of these methods mainly focus on leveraging extra monolin- gual data through back-translation (Sennrich et al., 2016) and self-training (He et al., 2020), or trans- lation knowledge transfer through parallel data in- volving other assisting language pairs (Firat et al., 2016a,b; Johnson et al., 2017; Neubig and Hu, 2018).1 Large scale pre-training is another recent trend to utilize large monolingual data for NMT (Liu et al., 2020). However, very few work has con- sidered low-resource NMT without using auxiliary data or other pivot languages. In the presence of a sufficient amount of in- domain monolingual data, back-translation (BT) has proved to be quite successful (Edunov et al., 2018). In this approach, a reverse intermediate …

Content truncated to 500 words