Speculative Beam Search for Simultaneous Translation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1395–1402, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1395 Speculative Beam Search for Simultaneous Translation Renjie Zheng 2,∗ Mingbo Ma 1,∗ Baigong Zheng 1 Liang Huang 1,2 1Baidu Research, Sunnyvale, CA, USA 2Oregon State University, Corvallis, OR, USA zrenj11@gmail.com {mingboma, baigongzheng, lianghuang}@baidu.com Abstract Beam search is universally used in full- sentence translation but its application to simultaneous translation remains non-trivial, where output words are committed on the fly. In particular, the recently proposed wait-k pol- icy (Ma et al., 2019a) is a simple and effec- tive method that (after an initial wait) com- mits one output word on receiving each input word, making beam search seemingly impos- sible. To address this challenge, we propose a speculative beam search algorithm that hal- lucinates several steps into the future in order to reach a more accurate decision, implicitly benefiting from a target language model. This makes beam search applicable for the first time to the generation of a single word in each step. Experiments over diverse language pairs show large improvements over previous work. 1 Introduction Beam search has been widely used in neural text generation such as machine translation (Sutskever et al., 2014; Bahdanau et al., 2014), summariza- tion (Rush et al., 2015; Ranzato et al., 2016), and image captioning (Vinyals et al., 2015; Xu et al., 2015). It often leads to substantial improvement over greedy search and becomes an essential com- ponent in almost all text generation systems. However, beam search is easy for the above tasks because they are all full-sequence problems, where the whole input sequence is available at the beginning and the output sequence only needs to be revealed in full at the end. By contrast, in lan- guage and speech processing, there are many in- cremental processing tasks with simultaneity re- quirements, where the output needs to be revealed to the user incrementally without revision (word by word, or in chunks) and the input is also being ∗These authors contributed equally. received incrementally. Two most salient exam- ples are streaming speech recognition (Chiu et al., 2018), widely used in speech input and dialog sys- tems (such as Siri), and simultaneous translation (Bangalore et al., 2012; Oda et al., 2015; Gris- som II et al., 2014; Jaitly et al., 2016), widely used in international conferences and negotiations. In these tasks, the use of full-sentence beam search becomes seemingly impossible as output words need to be committed on the fly. How to adapt beam search for such incremen- tal tasks in order to improve their generation qual- ity? We propose a general technique of specula- tive beam search (SBS), and apply it to simulta- neous translation. At a very high level, to gener- ate a single word, instead of simply choosing the highest-scoring one (as in greedy search), we fur- ther speculate w steps into the future, and use the ranking …

Content truncated to 500 words