Tagged End-to-End Simultaneous Speech Translation Training Using Simultaneous Interpretation Data

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 363–375 July 13-14, 2023 c⃝2023 Association for Computational Linguistics Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data Yuka Ko Ryo Fukuda Yuta Nishikawa Yasumasa Kano Katsuhito Sudoh Satoshi Nakamura Nara Institute of Science and Technology ko.yuka.kp2@is.naist.jp Abstract Simultaneous speech translation (SimulST) translates partial speech inputs incrementally. Although the monotonic correspondence be- tween input and output is preferable for smaller latency, it is not the case for distant language pairs such as English and Japanese. A prospec- tive approach to this problem is to mimic si- multaneous interpretation (SI) using SI data to train a SimulST model. However, the size of such SI data is limited, so the SI data should be used together with ordinary bilingual data whose translations are given in offline. In this paper, we propose an effective way to train a SimulST model using mixed data of SI and offline. The proposed method trains a single model using the mixed data with style tags that tell the model to generate SI- or offline-style outputs. Experiment results show improve- ments of BLEURT in different latency ranges, and our analyses revealed the proposed model generates SI-style outputs more than the base- line. 1 Introduction Simultaneous speech translation (SimulST) is a technique to translate speech incrementally without waiting for the end of a sentence. Since SimulST should work in small latency against the input speech, monotonic translation following the word order of the source language is preferable. How- ever, making translation monotonic is not trivial especially for distant language pairs with different word orders, such as English and Japanese. Most recent SimulST studies still use parallel corpora only with offline translations and potentially have the limitation to work in a monotonic way. A prospective approach to this problem is to use SI data to train a SimulST model for mimick- ing simultaneous interpretation (SI). There are sev- eral SI data resources developed so far for English- Japanese (Toyama et al., 2004; Shimizu et al., 2013; Doi et al., 2021). Despite these efforts, SI data are still very small compared to bilingual data based on offline translations. Using such scarce SI data to fine-tune an offline translation model causes over- fitting on the small SI data. Training a model using mixed data of offline and SI data is another option to mitigate the problem of data scarcity, but the simple data mixture causes confusion between the output styles of offline translation and SI. In this paper, we propose a method to train a SimulST model using mixed data of SI and offline translation with style tags to tell the model to gener- ate SI- or offline-style output selectively. It has the advantage of sharing two different styles in a single model and generating SI-style outputs by putting the SI-style tag in the decoding, which are lever- aged by offline translation data. Experiment results using MuST-C and small SI data showed improve- ments of BLEURT by the …

Content truncated to 500 words