Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5857–5864 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5857 Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings Junkun Chen 1⇤ Renjie Zheng 2⇤ Atsuhito Kita 1† Mingbo Ma 2 Liang Huang 1,2 1Oregon State University, Corvallis, OR, USA 2Baidu Research, Sunnyvale, CA, USA chenjun2@oregonstate.edu, renjiezheng@baidu.com Abstract Simultaneous translation is vastly different from full-sentence translation, in the sense that it starts translation before the source sentence ends, with only a few words delay. However, due to the lack of large-scale, high-quality simultaneous translation datasets, most such systems are still trained on conventional full- sentence bitexts. This is far from ideal for the simultaneous scenario due to the abundance of unnecessary long-distance reorderings in those bitexts. We propose a novel method that rewrites the target side of existing full- sentence corpora into simultaneous-style trans- lation. Experiments on Zh!En and Ja!En simultaneous translation show substantial im- provements (up to +2.7 BLEU) with the addi- tion of these generated pseudo-references. 1 Introduction Simultaneous translation, which starts translation before the source sentence ends, is substantially more challenging than full-sentence translation due to partial observation of the (incrementally re- vealed) source sentence. Recently, it has witnessed great progress thanks to fixed-latency policies (such as wait-k) (Ma et al., 2019) and adaptive policies (Gu et al., 2017; Arivazhagan et al., 2019). However, all state-of-the-art simultaneous trans- lation models are trained on conventional parallel text which involve many unnecessary long-distance reorderings (Birch et al., 2009; Braune et al., 2012); see Fig. 1 for an example. The simultaneous trans- lation models trained using these parallel sentences will learn to either make bold hallucinations (for fixed-latency policies) or introduce long delays (for adaptive ones). Alternatively, one may want to use transcribed corpora from professional simultaneous interpretation (Matsubara et al., 2002; Bendazzoli et al., 2005; Neubig et al., 2018). These data are more monotonic in word-order, but they are all very ⇤⇤Equal contribution. †Currently at Columbia University. zh¯onggu´o de x¯ıb`u yˇou hˇundu¯o g¯ao sh¯an Source Input -˝ Ñ 西Ë à⇢ ÿ q china ’s west have many big mountain Gold-Ref there are many big mountains in western china Pseudo-Refs (wait-1) china ’s west has many big mtns (...wait-2...) the chinese west has many big mtns (...wait-3...) western china has many big mtns (...wait-4...) there are many big ... Figure 1: Example of unnecessary reorderings in the bitext which can force the model to anticipate aggres- sively, along with the ideal pseudo-references with dif- ferent wait-k policies. Larger k improves fluency but sacrifices latency (pseudo-refs with k ≥ 4 are identical to the original reference). (mtns: mountains) small in size due to the high cost of data collection (e.g., the NAIST one (Neubig et al., 2018) has only 387k target words). More importantly, simultane- ous interpreters tend to summarize and inevitably make many mistakes (Shimizu et al., 2014; Xiong et al., 2019; Zheng et al., 2020) due to the high cognitive load and intense time pressure during …

Content truncated to 500 words