Learning Coupled Policies for Simultaneous Machine Translation using Imitation Learning

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2709–2719 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2709 Learning Coupled Policies for Simultaneous Machine Translation using Imitation Learning Philip Arthur† Department of Data Science and AI Monash University Trevor Cohn‡ School of Computing and Information Systems University of Melbourne †{philip.arthur, gholamreza.haffari}@monash.edu ‡tcohn@unimelb.edu.au Gholamreza Haffari† Department of Data Science and AI Monash University Abstract We present a novel approach to efficiently learn a simultaneous translation model with cou- pled programmer-interpreter policies. First, we present an algorithmic oracle to produce or- acle READ/WRITE actions for training bilin- gual sentence-pairs using the notion of word alignments. This oracle actions are designed to capture enough information from the par- tial input before writing the output. Next, we perform a coupled scheduled sampling to effec- tively mitigate the exposure bias when learning both policies jointly with imitation learning. Experiments on six language-pairs show our method outperforms strong baselines in terms of translation quality while keeping the transla- tion delay low. 1 Introduction Simultaneous machine translation (SIMT) is a set- ting where the translator needs to incrementally generate the translation while the source utterance is being received. This is a challenging transla- tion scenario as the SIMT model needs to trade off delaying translation output and the quality of the generated translation. Recent research on SIMT relies on a strategy to decide when to read a word from the input or write a word to the output (Satija and Pineau, 2016; Gu et al., 2017). This is based on a sequential decision making formulation of SIMT, where the decision making about the next READ/WRITE ac- tion is made by an agent, interacting with the neural machine translation (NMT) environment. Current approaches are sub-optimal as they either fix the agent’s policy to focus learning the NMT model (Ma et al., 2019; Dalvi et al., 2018) or learn adap- tive agent policies while the NMT model is fixed (Gu et al., 2017; Alinejad et al., 2018). We argue that the interpreter should also learn to generate correct translation from incomplete input informa- tion. This is challenging as we need to optimize both programmer’s and interpreter’s policies to bal- ance the tradeoff between quality and delay in the reward. Previous research has considered the use of im- itation learning (IL) to train the agent’s policy (Zheng et al., 2019a,b), which is generally supe- rior to reinforcement Learning (RL) in terms of the stability and sample complexity. However, the bottleneck of IL in SIMT is the unavailability of the oracle sequence of actions. Designing algorith- mic oracles to compute sequence of READ/WRITE actions with low translation latency and high trans- lation quality is under-explored. We present an IL approach to efficiently learn ef- fective coupled programmer-interpreter policies in SIMT, based on the following contributions. First, we present a simple, fast, and effective algorithmic oracle to produce oracle actions from the training bilingual sentence-pairs based on statistical word alignments (Brown et …

Content truncated to 500 words