Rewriter-Evaluator Architecture for Neural Machine Translation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5701–5710 August 1–6, 2021. ©2021 Association for Computational Linguistics 5701 Rewriter-Evaluator Architecture for Neural Machine Translation Yangming Li♠ and Kaisheng Yao♥ ♠Tencent AI Lab ♥Ant Group newmanli@tencent.com,kaisheng.yao@antgroup.com Abstract A few approaches have been developed to im- prove neural machine translation (NMT) mod- els with multiple passes of decoding. How- ever, their performance gains are limited be- cause of lacking proper policies to terminate the multi-pass process. To address this issue, we introduce a novel architecture of Rewriter- Evaluator. Translating a source sentence in- volves multiple rewriting passes. In every pass, a rewriter generates a new translation to im- prove the past translation. Termination of this multi-pass process is determined by a score of translation quality estimated by an evalua- tor. We also propose prioritized gradient de- scent (PGD) to jointly and efficiently train the rewriter and the evaluator. Extensive experi- ments on three machine translation tasks show that our architecture notably improves the per- formances of NMT models and significantly outperforms prior methods. An oracle exper- iment reveals that it can largely reduce perfor- mance gaps to the oracle policy. Experiments confirm that the evaluator trained with PGD is more accurate than prior methods in determin- ing proper numbers of rewriting. 1 Introduction Encoder-Decoder architecture (Sutskever et al., 2014) has been widely used in natural language generation, especially neural machine translation (NMT) (Bahdanau et al., 2015; Gehring et al., 2017; Vaswani et al., 2017; Zhang et al., 2019; Kitaev et al., 2020). Given a source sentence, an en- coder firstly converts it into hidden representations, which are then conditioned by a decoder to produce a target sentence. In analogy to the development of statistical machine translation (SMT) (Och and Ney, 2002; Shen et al., 2004; Zhang and Gildea, 2008), some recent methods in NMT attempt to im- prove the encoder-decoder architecture with multi- pass decoding (Xia et al., 2017; Zhang et al., 2018; Geng et al., 2018; Niehues et al., 2016). In these models, more than one translation is generated for a source sentence. Except for the first translation, each of the later translations is conditioned on the previous one. While these methods have achieved promising results, they lack a proper termination poqlicy for this multi-turn process. For instance, Xia et al. (2017); Zhang et al. (2018) adopt a fixed number of decoding passes, which is inflexible and can be sub-optimal. Geng et al. (2018) utilize reinforcement learning (RL) (Sutton et al., 2000) to automatically decide the number of decoding passes. However, RL is known to be unstable due to the high variance in gradient estimation (Boyan and Moore, 1995). To address this problem, we introduce a novel architecture, Rewriter-Evaluator. This architecture contains a rewriter and an evaluator. The trans- lation process involves multiple passes. Given a source sentence, at every turn, the rewriter gener- ates a new target sequence to improve the transla- tion from the …

Content truncated to 500 words