Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6775 - 6788 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework Shaolei Zhang 1,2, Yang Feng 1,2∗ 1Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) 2 University of Chinese Academy of Sciences, Beijing, China {zhangshaolei20z, fengyang}@ict.ac.cn Abstract Simultaneous machine translation (SiMT) starts translating while receiving the streaming source inputs, and hence the source sentence is always incomplete during translating. Different from the full-sentence MT using the conven- tional seq-to-seq architecture, SiMT often ap- plies prefix-to-prefix architecture, which forces each target word to only align with a partial source prefix to adapt to the incomplete source in streaming inputs. However, the source words in the front positions are always illusoryly con- sidered more important since they appear in more prefixes, resulting in position bias, which makes the model pay more attention on the front source positions in testing. In this pa- per, we first analyze the phenomenon of posi- tion bias in SiMT, and develop a Length-Aware Framework to reduce the position bias by bridg- ing the structural gap between SiMT and full- sentence MT. Specifically, given the stream- ing inputs, we first predict the full-sentence length and then fill the future source position with positional encoding, thereby turning the streaming inputs into a pseudo full-sentence. The proposed framework can be integrated into most existing SiMT methods to further improve performance. Experiments on two representa- tive SiMT methods, including the state-of-the- art adaptive policy, show that our method suc- cessfully reduces the position bias and thereby achieves better SiMT performance. 1 Introduction Simultaneous machine translation (SiMT) (Cho and Esipova, 2016; Gu et al., 2017; Ma et al., 2019; Arivazhagan et al., 2019) starts translating while receiving the streaming source inputs, which is cru- cial to many live scenarios, such as simultaneous interpretation, live broadcast and synchronized sub- titles. Compared with full-sentence machine trans- lation (MT) waiting for the complete source sen- ∗Corresponding author: Yang Feng. � � Source : Target : full sentence (a) Full-sentence MT with seq-to-seq architecture � � Source : Target : source prefix (b) SiMT with prefix-to-prefix architecture Figure 1: Architecture of full-sentence MT and SiMT. tence, SiMT is more challenging since the source sentence is always incomplete during translating. To process the incomplete source, SiMT has a different architecture from full-sentence MT, as shown in Figure 1. Full-sentence MT applies the seq-to-seq architecture (Sutskever et al., 2014), where each target word can be translated based on a complete source sentence. SiMT always applies prefix-to-prefix architecture (Ma et al., 2019) to force each target word to only align with a source prefix rather than the complete source sentence, where the source prefix consists of partial source words in the front position and is monotonically non-decreasing at each step. Although the prefix-to-prefix architecture effec- tively adapts to the streaming inputs by removing the subsequent …

Content truncated to 500 words