Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation

Proceedings of NAACL-HLT 2019, pages 2047–2053 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2047 Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation Weijia Xu University of Maryland weijia@cs.umd.edu Xing Niu University of Maryland xingniu@cs.umd.edu Marine Carpuat University of Maryland marine@cs.umd.edu Abstract Despite some empirical success at correcting exposure bias in machine translation, sched- uled sampling algorithms suffer from a ma- jor drawback: they incorrectly assume that words in the reference translations and in sam- pled sequences are aligned at each time step. Our new differentiable sampling algorithm ad- dresses this issue by optimizing the probabil- ity that the reference can be aligned with the sampled output, based on a soft alignment pre- dicted by the model itself. As a result, the output distribution at each time step is eval- uated with respect to the whole predicted se- quence. Experiments on IWSLT translation tasks show that our approach improves BLEU compared to maximum likelihood and sched- uled sampling baselines. In addition, our ap- proach is simpler to train with no need for sam- pling schedule and yields models that achieve larger improvements with smaller beam sizes.1 1 Introduction Neural machine translation (NMT) models are typically trained to maximize the likelihood of ref- erence translations (Sutskever et al., 2014; Bah- danau et al., 2015). While simple and effective, this objective suffers from the exposure bias prob- lem (Ranzato et al., 2015): the model is only ex- posed to reference target sequences during train- ing, but has to rely on its own predictions at infer- ence. As a result, errors can accumulate along the generated sequence at inference time. This is a well-known issue in sequential deci- sion making (Langford and Zadrozny, 2005; Co- hen and Carvalho, 2005; K¨a¨ari¨ainen and Lang- ford, 2006, i.a.) and it has been addressed in past work by incorporating the previous decod- ing choices into the training scheme, using imi- tation learning (Daum´e et al., 2009; Ross et al., 1The code is available at https://github.com/ Izecson/saml-nmt 2011; Bengio et al., 2015; Leblond et al., 2018) and reinforcement learning (Ranzato et al., 2015; Bahdanau et al., 2016) techniques. In this pa- per, we focus on a simple and computationally in- expensive family of approaches, known as Data as Demonstrator (Venkatraman et al., 2015) and scheduled sampling (Bengio et al., 2015; Goyal et al., 2017). The algorithms use a stochastic mix- ture of the reference words and model predictions with an annealing schedule controlling the mix- ture probability. Despite their empirical success in various sequence prediction tasks, they are based on an assumption that does not hold for machine translation: they assume that words in the refer- ence translations and in sampled sequences are aligned at each time step, which results in weak and sometimes misleading training signals. In this paper, we introduce a differentiable sam- pling algorithm that exposes machine translation models to their own predictions during training, and allows for differences in word order when comparing model outputs with …

Content truncated to 500 words