Dynamic Masking for Improved Stability in Online Spoken Language Translation

Dynamic Masking for Improved Stability in Online Spoken Language Translation Yuekun Yao ykyao.cs@gmail.com Barry Haddow bhaddow@staffmail.ed.ac.uk School of Informatics, University of Edinburgh, 10 Crichton Street, Edinburgh, Scotland Abstract For spoken language translation (SLT) in live scenarios such as conferences, lectures and meet- ings, it is desirable to show the translation to the user as quickly as possible, avoiding an an- noying lag between speaker and translated captions. In other words, we would like low-latency, online SLT. If we assume a pipeline of automatic speech recognition (ASR) and machine trans- lation (MT) then a simple but effective approach to online SLT is to pair an online ASR system, with a retranslation strategy, where the MT system retranslates every update received from ASR. However this can result in annoying “flicker” as the MT system updates its translation. A possible solution is to add a fixed delay, or “mask” to the the output of the MT system, but a fixed global mask re-introduces undesirable latency to the output. We introduce a method for dynamically determining the mask length, which provides a better latency–flicker trade-off curve than a fixed mask, without affecting translation quality. 1 Introduction A common approach to Spoken Language Translation (SLT) is to use a cascade (or pipeline) consisting of automatic speech recognition (ASR) and machine translation (MT). In a live trans- lation setting, such as a lecture or conference, we would like the transcriptions or translations to appear as quickly as possible, so that they do not “lag” noticeably behind the speaker. In other words, we wish to minimise the latency of the system. Many popular ASR toolkits can operate in an online mode, where the transcription is produced incrementally, instead of waiting for the speaker to finish their utterance. However online MT is less well supported, and is complicated by the reordering which is often necessary in translation, and by the use of encoder-decoder models which assume sight of the whole source sentence. Some systems for online SLT rely on the streaming approach to translation, perhaps in- spired by human interpreters. In this approach, the MT system is modified to translate incre- mentally, and on each update from ASR it will decide whether to update its translation, or wait for further ASR output (Cho and Esipova, 2016; Ma et al., 2019; Zheng et al., 2019a,b; Arivazhagan et al., 2019). The difficulty with the streaming approach is that the system has to choose between com- mitting to a particular choice of translation output, or waiting for further updates from ASR, and does not have the option to revise an incorrect choice. Furthermore, all the streaming approaches referenced above require specialised training of the MT system, and modified infer- ence algorithms. To address the issues above, we construct our online SLT system using the retransla- tion approach (Niehues et al., 2018; Arivazhagan et al., 2020a), which is less studied but Proceedings of the 14th Conference of the Association for Machine Translation in the Americas October 6 - 9, 2020, Volume 1: …

Content truncated to 500 words