Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4528–4537 August 1–6, 2021. ©2021 Association for Computational Linguistics 4528 Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models Sumanta Bhattacharyya, Amirmohammad Rooshenas∗ Department of Computer Science, College of Computing and Informatics University of North Carolina Charlotte {sbhatta9,rooshenas}@uncc.edu Subhajit Naskar, Simeng Sun, Mohit Iyyer, and Andrew McCallum College of Information and Computer Science, University of Massachusetts Amherst {snaskar,simeng,miyyer,mccallum}@cs.umass.edu Abstract The discrepancy between maximum likeli- hood estimation (MLE) and task measures such as BLEU score has been studied before for autoregressive neural machine translation (NMT) and resulted in alternative training al- gorithms (Ranzato et al., 2016; Norouzi et al., 2016; Shen et al., 2016; Wu et al., 2018). How- ever, MLE training remains the de facto ap- proach for autoregressive NMT because of its computational efficiency and stability. Despite this mismatch between the training objective and task measure, we notice that the samples drawn from an MLE-based trained NMT sup- port the desired distribution – there are sam- ples with much higher BLEU score compar- ing to the beam decoding output. To bene- fit from this observation, we train an energy- based model to mimic the behavior of the task measure (i.e., the energy-based model assigns lower energy to samples with higher BLEU score), which is resulted in a re-ranking algo- rithm based on the samples drawn from NMT: energy-based re-ranking (EBR). We use both marginal energy models (over target sentence) and joint energy models (over both source and target sentences). Our EBR with the joint en- ergy model consistently improves the perfor- mance of the Transformer-based NMT: +3.7 BLEU points on IWSLT’14 German-English, +3.37 BELU points on Sinhala-English, +1.4 BLEU points on WMT’16 English-German tasks. 1 Introduction Autoregressive models are widely used for neural machine translation (NMT) (Bahdanau et al., 2015; Gehring et al., 2017; Vaswani et al., 2017). The autoregressive factorization provides a tractable likelihood computation as well as efficient sam- pling. The former results in the effective maxi- mum likelihood estimation (MLE) for training the ∗Amirmohammad Rooshenas is the corresponding author. parameters of NMT models. However, optimiz- ing likelihood does not guarantee an improvement in task-based measures such as the BLEU score, which has motivated directly optimizing task mea- sures with reinforcement learning (Ranzato et al., 2016; Norouzi et al., 2016; Shen et al., 2016; Bah- danau et al., 2017; Wu et al., 2018). However, for NMT, these training algorithms are often used in conjunction with MLE training (Wu et al., 2018) or as fine-tuning (Choshen et al., 2020). Interestingly, we observe that samples drawn from an NMT model trained using MLE may have higher quality (measured with BLEU) than the out- puts of beam search. In particular, we draw 100 target samples for each source sentence from an NMT model trained using MLE on the IWSLT’14 German-English task, and observe that an ora- cle ranker – i.e. argmaxy∼PNMT(y|x) BLEU(., y∗), where (x, y∗) is …

Content truncated to 500 words