Better Simultaneous Translation with Monotonic Knowledge Distillation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2334–2349 July 9-14, 2023 ©2023 Association for Computational Linguistics Better Simultaneous Translation with Monotonic Knowledge Distillation Shushu Wang 1, Jing Wu 2, Kai Fan 2, Wei Luo 2, Jun Xiao 1, Zhongqiang Huang 2 1Zhejiang University ,2 Alibaba DAMO Academy {wangshushu0213, junx}@zju.edu.cn {wj334275, k.fan, muzhuo.lw, z.huang}@alibaba-inc.com Abstract Simultaneous machine translation (SiMT) presents a unique challenge as it requires gen- erating target tokens before the source sentence is fully consumed. This can lead to the halluci- nation problem, where target tokens are gener- ated without support from the source sentence. The prefix-to-prefix training data used to train SiMT models are not always parallel, due to divergent word order between the source and target languages, and can contribute to the prob- lem. In this paper, we propose a novel approach that leverages traditional translation models as teachers and employs a two-stage beam search algorithm to generate monotonic yet accurate reference translations for sequence-level knowl- edge distillation. Experimental results demon- strate the significant improvements achieved by our approach over multiple strong SiMT baselines, leading to new state-of-the-art perfor- mance across various language pairs. Notably, when evaluated on a monotonic version of the WMT15 De→En test set, which includes refer- ences generated in a more monotonic style by professional translators, our approach achieves even more substantial improvement over the baselines. The source code and data are pub- licly available for further exploration1. 1 Introduction Simultaneous machine translation (SiMT) starts to translate with only a partial observation of the source sentence and can present unique challenges compared to full-sentence translation, particularly when employing offline NMT models. Prefix-to- prefix (P2P) methods such as the wait-k policy (Ma et al., 2019a) have been developed to narrow the gap between training and inference. However, these methods inherently rely on parallelism at the prefix level, which may not always be present in conventional parallel text. 1https://github.com/wangshushu0213/ Monotonic-Translation-Generation 韩国 对 美 出口 则 下滑 11.8%，因 石油 产量 下滑。 Since the sales of oil products declined, Korean exports to U.S. decreased by 11.8%. South Korea’s exports to U.S. fell 11.8% as sales of oil fell. source prefix reference prefix monotonic reference prefix Figure 1: An example of a parallel sentence pair, with color-coded parallel clauses. The boxes highlight the prefixes selected based on a wait-3 approach. Trainset k = 1 k = 3 k = 5 k = 7 k = 9 WMT15 De→En 30.4 15.2 8.5 5.1 3.3 CWMT19 Zh→En 25.4 12 6.3 3.6 2.1 IWSLT15 En→Vi 17.3 5.2 1.9 0.8 0.4 Table 1: Anticipation rates (AR%) of the original train- ing sets, measuring the percentage of target tokens with a reordering distance ≥ k (see definition in Appendix B). The parallel text utilized for training offline MT models exhibits a wide range of word reordering between the source and target languages, resulting in non-parallel prefix-to-prefix pairs, as depicted in Figure 1. Table 1 highlights the challenge faced by a wait-k model, which …

Content truncated to 500 words