Towards Enhancing Faithfulness for Neural Machine Translation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2675–2684, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2675 Towards Enhancing Faithfulness for Neural Machine Translation Rongxiang Weng1, Heng Yu1, Xiangpeng Wei2,3∗, Weihua Luo1 1Machine Intelligence Technology Lab, Alibaba Group, China 2Institute of Information Engineering, Chinese Academy of Sciences, China 3School of Cyber Security, University of Chinese Academy of Sciences, China {wengrx,yuheng.yh}@alibaba-inc.com weixiangpeng@iie.ac.cn weihua.luowh@alibaba-inc.com Abstract Neural machine translation (NMT) has achieved great success due to the ability to generate high-quality sentences. Compared with human translations, one of the drawbacks of current NMT is that translations are not usually faithful to the input, e.g., omitting information or generating unrelated fragments, which inevitably decreases the overall quality, especially for human readers. In this paper, we propose a novel training strategy with a multi-task learning paradigm to build a faithfulness enhanced NMT model (named FENMT). During the NMT training process, we sample a subset from the training set and translate them to get fragments that have been mistranslated. Afterward, the proposed multi-task learning paradigm is employed on both encoder and decoder to guide NMT to correctly translate these fragments. Both automatic and human evaluations verify that our FENMT could improve translation quality by effectively reducing unfaithful translations. 1 Introduction Neural machine translation (NMT) based on the encoder-decoder framework (Sutskever et al., 2014; Cho et al., 2014; Bahdanau et al., 2014; Luong et al., 2015b) has obtained state-of-the-art per- formance on many language pairs (Wu et al., 2019; Wei et al., 2020). Various neural archi- tectures have been explored for modeling NMT under this framework, such as recurrent neural network (RNN) (Bahdanau et al., 2014; Luong et al., 2015b, RNNSearch), convolutional neu- ral network (CNN) (Gehring et al., 2016, Conv- S2S) and self-attention network (Vaswani et al., 2017, Transformer). Compared with human trans- lations or traditional statistical machine translation (SMT) (Koehn et al., 2007b; Chiang, 2007), NMT ∗Work done during the internship at Alibaba Group. can generate high-quality sentences that are very close to natural language. However, it usually ap- pears some parts (e.g., phrase) from input sentences cannot be correctly translated, leading to that the translation is inadequate for direct using in some scenarios. This phenomenon appeals that enhanc- ing the faithfulness of translations is an important aspect for further improving NMT. We summarize three possible causes for the un- faithfulness problem based on the encoder-decoder framework: 1). Some parts from input sentences are hard to encode, and thus cannot be translated correctly. 2). The decoder cannot fetch the cor- rect contextual representation from the encoder. 3).The dominant language model of NMT prompts the decoder generates common words to make sure outputs are fluent. Several recent studies are pro- posed following one of the above perspectives and have achieved considerable effects. Zheng et al. (2019) proposed to divide the encoder output into past and future parts to fine-grained modeling con- textual representation. Feng et al. (2020) proposed a faithfulness part to optimize the contextual rep- resentation before feeding into …

Content truncated to 500 words