Learning to Stop in Structured Prediction for Neural Machine Translation

Proceedings of NAACL-HLT 2019, pages 1884–1889 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1884 Learning to Stop in Structured Prediction for Neural Machine Translation Mingbo Ma ∗†,‡ Renjie Zheng ∗ ‡,† Liang Huang †,‡ †Baidu Research, Sunnyvale, CA ‡School of EECS, Oregon State University, Corvallis, OR {cosmmb, zrenj11, liang.huang.sh}@gmail.com Abstract Beam search optimization (Wiseman and Rush, 2016) resolves many issues in neural machine translation. However, this method lacks principled stopping criteria and does not learn how to stop during training, and the model naturally prefers longer hypotheses dur- ing the testing time in practice since they use the raw score instead of the probability-based score. We propose a novel ranking method which enables an optimal beam search stop- ping criteria. We further introduce a struc- tured prediction loss function which penalizes suboptimal finished candidates produced by beam search during training. Experiments of neural machine translation on both synthetic data and real languages (German→English and Chinese→English) demonstrate our pro- posed methods lead to better length and BLEU score. 1 Introduction Sequence-to-sequence (seq2seq) models based on RNNs (Sutskever et al., 2014; Bahdanau et al., 2014), CNNs (Gehring et al., 2017) and self- attention (Vaswani et al., 2017) have achieved great successes in Neural Machine Translation (NMT). The above family of models encode the source sentence and predict the next word in an autoregressive fashion at each decoding time step. The classical “cross-entropy” training objective of seq2seq models is to maximize the likelihood of each word in the translation reference given the source sentence and all previous words in that ref- erence. This word-level loss ensures efficient and scalable training of seq2seq models. However, this word-level training objective suf- fers from a few crucial limitations, namely the la- bel bias, the exposure bias, and the loss-evaluation mismatch (Lafferty et al., 2001; Bengio et al., ∗ Equal contribution BSO BSO This work This work Figure 1: The BLEU score of BSO decreases after beam size 3 as results of increasing length ratio1in German→English translation. Our model gets higher BLEU with larger beam. 2015a; Venkatraman et al., 2015). In addition, more importantly, at decoding time, beam search is universally adopted to improve the search qual- ity, while training is fundamentally local and greedy. Several researchers have proposed differ- ent approaches to alleviate above problems, such as reinforcement learning-based methods (Ran- zato et al., 2016; Rennie et al., 2017; Zheng et al., 2018b), training with alternative references (Shen et al., 2016; Zheng et al., 2018a). Recently, Wise- man and Rush (2016) attempt to address these issues with a structured training method, Beam Search Optimization (BSO). While BSO outper- forms other proposed methods on German-to- English translation, it also brings a different set of problems as partially discussed in (Ma, 2018) which we present with details below. 1There are two types of “length ratios” in this paper: (a) target to reference ratio (|y|/|y∗|), which is used in BLEU, and (b) target to source ratio (|y|/|x|). By default, the term “length ratio” …

Content truncated to 500 words