NAIST Simultaneous Speech-to-Text Translation System for IWSLT 2022

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 286 - 292 May 26-27, 2022 c⃝2022 Association for Computational Linguistics NAIST Simultaneous Speech-to-Text Translation System for IWSLT 2022 Ryo Fukuda†, Yuka Ko†, Yasumasa Kano†, Kosuke Doi†, Hirotaka Tokuyama†, Sakriani Sakti†‡, Katsuhito Sudoh†, Satoshi Nakamura† †Nara Institute of Science and Technology, Japan ‡Japan Advanced Institute of Science and Technology, Japan fukuda.ryo.fo3@is.naist.jp Abstract This paper describes NAIST’s simultane- ous speech translation systems developed for IWSLT 2022 Evaluation Campaign. We partic- ipated the speech-to-speech track for English- to-German and English-to-Japanese. Our pri- mary submissions were end-to-end systems us- ing adaptive segmentation policies based on Prefix Alignment. 1 Introduction This paper describes NAIST’s submissions to IWSLT 2022 (Anastasopoulos et al., 2022) Simul- taneous Speech Translation track. We participated the speech-to-speech track for English-to-German (En-De) and English-to-Japanese (En-Ja) using our end-to-end simultaneous machine translation (SimulMT) systems. SimulMT based on neural machine translation (NMT) has achieved a large success in recent years. There are two different SimulMT approaches de- pending on the policy that determines READ (wait- ing for speech input) and WRITE (writing text out- put) actions: fixed and adaptive. Fixed policies are usually implemented by simple rules (Dalvi et al., 2018; Ma et al., 2019; Fukuda et al., 2021; Sen et al., 2021). They are simple yet often effective, but they sometimes make inappropriate decisions due to large word order differences, pauses, and so on. In contrast, adaptive policies decide READ or WRITE actions flexibly taking current context into account (Zheng et al., 2019a,b, 2020; Liu et al., 2021). They can be more effective than fixed policies in end-to-end speech-to-speech SimulMT because it is difficult to define fixed policies for speech input. In our systems, we use Bilingual Prefix Align- ment (Kano et al., 2022), which extracts alignment between bilingual prefix pairs in the training time, for prefix-to-prefix translation in SimulMT. The Bilingual Prefix Alignment is applied to extract Step 1 I Step 2 I bought Step 3 I bought a Step 4 I bought a pen 0.9 > 0.5 私は 0.2 < 0.5 0.3 < 0.5 0.7 > 0.5 私はペンを買った Read source words Boundary Prediction translation Step 5 I bought a pen . 0.7 > 0.5 私はペンを買った。 Figure 1: A brief overview of our prefix-to-prefix trans- lation process (Kano et al., 2022) from English to Japanese. The threshold of boundary probability is 0.5 in this example. Underlined parts are the forced output prefixes. prefix pairs of source language speech and target language translations. We also use the prefix pairs to train a boundary prediction model for an adaptive speech segmentation policy. Our system showed some improvements against wait-k baselines on the development data, in all the latency regimes in both En-De and En-Ja. 2 Simultaneous Speech Translation based on Bilingual Prefix Alignment We developed simultaneous speech translation (SimulST) based on offline speech translation (ST). Our SimulST system translates an incrementally- growing source language speech prefix into the tar- get language. When the system detects a segment boundary in source …

Content truncated to 500 words