HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1717–1733 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1717 HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints Sahana Ramnath, Melvin Johnson∗, Abhirut Gupta∗, Aravindan Raghuveer Google Research {sahanaramnath, melvinp, abhirut, araghuveer}@google.com Abstract Back-translation (BT) of target monolingual corpora is a widely used data augmentation strategy for neural machine translation (NMT), especially for low-resource language pairs. To improve effectiveness of the available BT data, we introduce HintedBT—a family of tech- niques which provides hints (through tags) to the encoder and decoder. First, we propose a novel method of using both high and low qual- ity BT data by providing hints (as source tags on the encoder) to the model about the qual- ity of each source-target pair. We don’t fil- ter out low quality data but instead show that these hints enable the model to learn effec- tively from noisy data. Second, we address the problem of predicting whether a source to- ken needs to be translated or transliterated to the target language, which is common in cross- script translation tasks (i.e., where source and target do not share the written script). For such cases, we propose training the model with additional hints (as target tags on the decoder) that provide information about the operation required on the source (translation or both translation and transliteration). We conduct experiments and detailed analyses on standard WMT benchmarks for three cross- script low/medium-resource language pairs: {Hindi,Gujarati,Tamil}→English. Our meth- ods compare favorably with five strong and well established baselines. We show that us- ing these hints, both separately and together, significantly improves translation quality and leads to state-of-the-art performance in all three language pairs in corresponding bilin- gual settings. 1 Introduction Neural machine translation (NMT) (Kalchbren- ner and Blunsom, 2013; Bahdanau et al., 2014; Sutskever et al., 2014; Wu et al., 2016; Hassan et al., 2018) models have become the state-of-the-art ap- proach to machine translation. However, NMT ∗*equal contribution Figure 1: Examples from the WMT 2014 Hindi→English test set. The top example is a case of only translation, and the bottom one is a case where some words in the source (a named-entity "Coulson", and English words ‘phone’, ‘hacking’ written in Hindi) need to be transliterated. models are data hungry and have been shown to under-perform in low-resource scenarios (Koehn and Knowles, 2017). Various supervised and unsu- pervised techniques (Song et al., 2019; Gulcehre et al., 2015) have been proposed to address the paucity of high-quality parallel data in such cases. Back-translation (Sennrich et al., 2016b) is one such widely used data augmentation technique in which synthetic parallel data is created by translat- ing monolingual data in the target language to the source language using a baseline system. However, in order to get high quality parallel back-translated (BT) data, we need a high quality target→source translation model (Burlot and Yvon, 2019). This in turn depends on having a substantial amount of high quality parallel (bitext) data already avail- able. For …

Content truncated to 500 words