Anticipation-Free Training for Simultaneous Machine Translation

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 43 - 61 May 26-27, 2022 c⃝2022 Association for Computational Linguistics Anticipation-Free Training for Simultaneous Machine Translation Chih-Chiang Chang, Shun-Po Chuang, Hung-yi Lee National Taiwan University {r09922057,f04942141,hungyilee}@ntu.edu.tw Abstract Simultaneous machine translation (SimulMT) speeds up the translation process by starting to translate before the source sentence is com- pletely available. It is difficult due to lim- ited context and word order difference between languages. Existing methods increase latency or introduce adaptive read-write policies for SimulMT models to handle local reordering and improve translation quality. However, the long-distance reordering would make the SimulMT models learn translation mistakenly. Specifically, the model may be forced to predict target tokens when the corresponding source tokens have not been read. This leads to aggres- sive anticipation during inference, resulting in the hallucination phenomenon. To mitigate this problem, we propose a new framework that de- compose the translation process into the mono- tonic translation step and the reordering step, and we model the latter by the auxiliary sorting network (ASN). The ASN rearranges the hid- den states to match the order in the target lan- guage, so that the SimulMT model could learn to translate more reasonably. The entire model is optimized end-to-end and does not rely on ex- ternal aligners or data. During inference, ASN is removed to achieve streaming. Experiments show the proposed framework could outper- form previous methods with less latency. 1 Introduction Simultaneous machine translation (SimulMT) is an extension of neural machine translation (NMT), aiming to perform streaming translation by out- putting the translation before the source input has ended. It is more applicable to real-world scenarios such as international conferences, where people could communicate fluently without delay. However, SimulMT faces additional difficul- ties compared to full-sentence translation – such a model needs to translate with limited context, and the different word order between languages would Training Target Reordered Output 他 在下午 打了個盹 他 在下午 打了個盹 (He) (in the afternoon) (took a nap) Output Input 他 在下午 打了個盹 (He) (in the afternoon) (took a nap) He took a nap in the afternoon Monotonic Translation Reordering Inference Stage Figure 1: Illustration of the training process. The trans- lated output is rearranged to match the order of training target, reducing anticipation. We use the gray part dur- ing inference. make streaming models learn translation mistak- enly. The problems can often be alleviated by in- creasing the context. Using more context allows the model to translate with more information, trad- ing off speed for quality. But the word order could be very different among languages. Increasing the context could only solve the local reordering prob- lem. If long-distance reordering exists in training data, the model would be forced to predict tokens in the target language when the corresponding source tokens have not been read. this is called anticipa- tion (Ma et al., 2019). Ignoring the long-distance reordering may cause unnecessarily high latency, or encourage aggressive anticipation, resulting in the hallucination phenomenon (Müller et …

Content truncated to 500 words