A Generative Framework for Simultaneous Machine Translation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6697–6706 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6697 A Generative Framework for Simultaneous Machine Translation Yishu Miao Imperial College London ByteDance ym713@ic.ac.uk Phil Blunsom Universtiy of Oxford DeepMind phil.blunsom@cs.ox.ac.uk Lucia Specia Imperial College London University of Sheffield l.specia@ic.ac.uk Abstract We propose a generative framework for si- multaneous machine translation. Conventional approaches use a fixed number of source words to translate or learn dynamic policies for the number of source words by reinforce- ment learning. Here we formulate simulta- neous translation as a structural sequence-to- sequence learning problem. A latent variable is introduced to model read or translate ac- tions at every time step, which is then inte- grated out to consider all the possible trans- lation policies. A re-parameterised Poisson prior is used to regularise the policies which allows the model to explicitly balance trans- lation quality and latency. The experiments demonstrate the effectiveness and robustness of the generative framework, which achieves the best BLEU scores given different average translation latencies on benchmark datasets. 1 Introduction The fundamental challenge of simultaneous ma- chine translation (SiMT) is the balance between the translation quality and the latency. It is non-trivial to find an optimal translation strategy, as there is generally a rivalry between the two objectives, i.e. reading more source words before translating leads to better translation quality, but it in turn results in higher latency due to the longer time for reading. Conventional Wait-k policies (Ma et al., 2019) put a hard limitation over the buffer size k1, which guarantees low latency but weakens flexibility and scalability when handling long and complicated language pairs. Alternatively, reinforcement learn- ing (RL) approaches (Gu et al., 2017; Satija and Pineau, 2016; Arthur et al., 2021) learn a dynamic policy using a combined reward of a quality metric like the BLEU score and AL (average lagging)2. 1The number of read source words minus the number of translated target words. 2A metric for evaluating translation latency by how many words have been read on average before translating a word. 𝑋1 𝑌1 2 3 4 2 3 4 5 (a) Wait-k (k=3). 𝑋1 𝑌1 2 3 4 2 3 4 5 (b) Adaptive wait-k (k=3). 𝑋1 𝑌1 2 3 4 2 3 4 5 (c) RL method. 𝑋1 𝑌1 2 3 4 2 3 4 5 (d) GSiMT. Figure 1: Example of translation paths of different si- multaneous translation models. However, the poor sample efficiency make it very difficult to learn a robust SiMT model with RL. In this paper we propose a generative framework with a latent variable that dynamically decides be- tween the actions of read or translate at every time step, enabling the formulation of SiMT as a struc- tural sequence-to-sequence learning task. Figure 1 depicts the examples of possible translation paths of different models. Wait-k only explores one hy- pothesis, while adaptive wait-k ensembles the other hypothesises with lower k. However, the hypoth- esises of reading more than …

Content truncated to 500 words