Self-training Reduces Flicker in Retranslation-based Simultaneous Translation

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3734–3744 May 2-6, 2023 ©2023 Association for Computational Linguistics Self-training Reduces Flicker in Retranslation-based Simultaneous Translation Sukanta Sen1∗ Rico Sennrich2,1 Biao Zhang1 Barry Haddow1 1School of Informatics, University of Edinburgh 2Department of Computational Linguistics, University of Zurich ssen@ed.ac.uk, sennrich@cl.uzh.ch, b.zhang@ed.ac.uk, bhaddow@ed.ac.uk Abstract In simultaneous translation, the retranslation approach has the advantage of requiring no modifications to the inference engine. How- ever, in order to reduce the undesirable flicker in the output, previous work has resorted to increasing the latency through masking, and introducing specialised inference, thus losing the simplicity of the approach. In this work, we show that self-training improves the flicker- latency tradeoff, while maintaining similar translation quality to the original. Our analysis indicates that self-training reduces flicker by controlling monotonicity. Furthermore, self- training can be combined with biased beam search to further improve the flicker-latency tradeoff. 1 Introduction Simultaneous machine translation systems, which process their input word by word instead of sen- tence by sentence, must strike a balance between producing output immediately (and so reducing quality because of incomplete input) and waiting for further input (and so increasing latency). Ide- ally, a good simultaneous translation system will provide a pareto-optimal tradeoff between quality and latency. A straightforward way of doing simul- taneous translation is retranslation (Niehues et al., 2016), which has the advantage that it can be used with an unmodified machine translation (MT) in- ference engine, and can perform better than the alternative streaming-based approaches (Arivazha- gan et al., 2020b). The disadvantage is that retrans- lation may change previous output causing flicker, leading to a poor user experience, and so flicker needs to be balanced with latency and quality. We argue that flickering is caused by two dif- ferent (but related) issues: (i) lexical instability of the translation – the system “changes its mind” as more source is revealed, swapping one word for ∗Work done while at the University of Edinburgh. another1 and (ii) non-monotonicity of the transla- tion – the system favours a non-monotonic trans- lation, which means it needs high latency in or- der to avoid flicker. Some of this instability and non-monotonicity is necessary – forced by syntac- tic differences between source and target, and lack of information in the prefixes – but some is due to arbitrary choices of the model. We aim to reduce these as far as possible. In non-autoregressive translation (NAT), a re- lated problem, known as the “multimodality” prob- lem (Gu et al., 2018), has been addressed using knowledge distillation (Kim and Rush, 2016, KD). We therefore investigate whether this can also re- duce flicker in simultaneous translation. Since the initial model and the distilled model have the same architecture in our work, approximating KD is essentially self-training2. We show that a self- trained model is able to achieve the same qual- ity as the initial model, but with improved flicker- latency tradeoff. We also show that self-training (Arivazhagan et al., 2020a) …

Content truncated to 500 words