On the Language Coverage Bias for Neural Machine Translation

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4778–4790 August 1–6, 2021. ©2021 Association for Computational Linguistics 4778 On the Language Coverage Bias for Neural Machine Translation Shuo Wang⋆ Zhaopeng Tu⊤ Zhixing Tan⋆ Shuming Shi⊤ Maosong Sun⋆† Yang Liu⋆†‡ ⋆Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, Tsinghua University ⊤Tencent AI Lab †Beijing Academy of Artificial Intelligence ‡Institute for AIR, Tsinghua University ⋆{wangshuo2018, zxtan, sms, liuyang2011}@tsinghua.edu.cn ⊤{zptu, shumingshi}@tencent.com Abstract Language coverage bias, which indicates the content-dependent differences between sen- tence pairs originating from the source and tar- get languages, is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice. By carefully designing experiments, we provide comprehensive analyses of the lan- guage coverage bias in the training data, and find that using only the source-original data achieves comparable performance with using full training data. Based on these observations, we further propose two simple and effective approaches to alleviate the language coverage bias problem through explicitly distinguishing between the source- and target-original train- ing data, which consistently improve the per- formance over strong baselines on six WMT20 translation tasks. Complementary to the trans- lationese effect, language coverage bias pro- vides another explanation for the performance drop caused by back-translation (Marie et al., 2020). We also apply our approach to both back- and forward-translation and find that mitigating the language coverage bias can im- prove the performance of both the two repre- sentative data augmentation methods and their tagged variants (Caswell et al., 2019). 1 Introduction In recent years, there has been a growing interest in investigating the effect of original languages in parallel data on neural machine translation (Bar- rault et al., 2020; Edunov et al., 2020; Marie et al., 2020). Several studies have shown that target- original test examples1 can lead to distortions in automatic and human evaluations, which should be omitted from machine translation test sets (Bar- rault et al., 2019; Zhang and Toral, 2019; Graham 1Target-original test examples are sentence pairs that are translated from the target language into the source language. government police US back world group President right way percent well old American official think show work public Trump women long school country rate next high company court going case see States home change team need House found market city children business part former point law good issue National another news white military life used United cent leader left election service member men number best black local called family put student chief game decision job later health came killed end statement support war officer call man better record live never start past office University ago London media plan term head great system area find across drug hand minister run companies lot look campaign hour today give Republican economic political away book Obama things child cost top likely Department early cut added really interest half security might move tax least night set feel got far Clinton others line friend death though times attack rule …

Content truncated to 500 words