Confidence-Aware Scheduled Sampling for Neural Machine Translation

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2327–2337 August 1–6, 2021. ©2021 Association for Computational Linguistics 2327 Confidence-Aware Scheduled Sampling for Neural Machine Translation Yijin Liu1∗, Fandong Meng2, Yufeng Chen1 Jinan Xu1†, and Jie Zhou2 1Beijing Jiaotong University, China 2Pattern Recognition Center, WeChat AI, Tencent Inc, China adaxry@gmail.com {fandongmeng, withtomzhou}@tencent.com {jaxu,chenyf}@bjtu.edu.cn Abstract Scheduled sampling is an effective method to alleviate the exposure bias problem of neu- ral machine translation. It simulates the in- ference scene by randomly replacing ground- truth target input tokens with predicted ones during training. Despite its success, its crit- ical schedule strategies are merely based on training steps, ignoring the real-time model competence, which limits its potential perfor- mance and convergence speed. To address this issue, we propose confidence-aware sched- uled sampling. Specifically, we quantify real- time model competence by the confidence of model predictions, based on which we design fine-grained schedule strategies. In this way, the model is exactly exposed to predicted to- kens for high-confidence positions and still ground-truth tokens for low-confidence posi- tions. Moreover, we observe vanilla sched- uled sampling suffers from degenerating into the original teacher forcing mode since most predicted tokens are the same as ground-truth tokens. Therefore, under the above confidence- aware strategy, we further expose more noisy tokens (e.g., wordy and incorrect word order) instead of predicted ones for high-confidence token positions. We evaluate our approach on the Transformer and conduct experiments on large-scale WMT 2014 English-German, WMT 2014 English-French, and WMT 2019 Chinese-English. Results show that our ap- proach significantly outperforms the Trans- former and vanilla scheduled sampling on both translation quality and convergence speed. 1 Introduction Neural Machine Translation (NMT) has made promising progress in recent years (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017). ∗ This work was done when Yijin Liu was interning at Pattern Recognition Center, WeChat AI, Tencent Inc, China †Jinan Xu is the corresponding author of the paper. Generally, NMT models are trained to maximize the likelihood of the next token given previous golden tokens as inputs, i.e., teacher forcing (Good- fellow et al., 2016). However, at the inference stage, golden tokens are unavailable. The model is exposed to an unseen data distribution generated by itself. This discrepancy between training and inference is named as the exposure bias problem (Ranzato et al., 2016). Many techniques have been proposed to allevi- ate the exposure bias problem. To our knowledge, they mainly fall into two categories. The one is sentence-level training, which treats the sentence- level metric (e.g., BLEU) as a reward, and directly maximizes the expected rewards of generated se- quences (Ranzato et al., 2016; Shen et al., 2016; Rennie et al., 2017). Although intuitive, they gen- erally suffer from slow and unstable training due to the high variance of policy gradients and the credit assignment problem (Sutton, 1984; Liu et al., 2018; Wang et al., 2018). Another category is sampling- based approaches, aiming to simulate the data dis- tribution of reference during training. Scheduled sampling …

Content truncated to 500 words