Online Sentence Segmentation for Simultaneous Interpretation using Multi-Shifted Recurrent Neural Network

Online Sentence Segmentation for Simultaneous Interpretation using Multi-Shifted Recurrent Neural Network Xiaolin Wang Masao Utiyama Eiichiro Sumita Advanced Translation Research and Development Promotion Center National Institute of Information and Communications Technology, Japan {xiaolin.wang,mutiyama,eiichiro.sumita}@nict.go.jp Abstract This paper is devoted to developing a re- current neural network (RNN) solution for segmenting the unpunctuated transcripts generated by automatic speech recognition for simultaneous interpretation. RNNs are effective in capturing long-distance depen- dencies and straightforward for online de- coding. Thus, they are ideal for the task compared to the conventional n-gram lan- guage model (LM) based approaches and recent neural machine translation based approaches. This paper proposes a multi- shifted RNN to address the trade-off be- tween accuracy and latency, which is one of the key characteristics of the task. Ex- periments show that our proposed method improves the segmentation accuracy mea- sured in F1 by 21.1% while maintains ap- proximately the same latency, and reduces the BLEU loss to the oracle segmenta- tion by 28.6%, when compared to a strong baseline of the RNN LM-based method. Our online sentence segmentation toolkit is open-sourced 1 to promote the field. 1 Introduction Simultaneous interpretation (SI) is to translate one spoken language into another spoken language in real time. Automated SI typically requires inte- grating two fundamental natural language process- ing technologies – automatic speech recognition (ASR) and machine translation (MT). Both tech- nologies have become quite capable after half a c⃝ 2019 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. 1https://github.com/arthurxlw/cytonNss even cats were watching this video cats were watching other cats watch this video but what ’s important here is the creativity that it inspired amongst this techie geeky internet culture there were remixes someone made an old timey ver- sion and then it went international there were remixes someone made an old timey version Table 1: Illustration of Input for Sentence Segmentation century’s intensive study, but one problem makes it difficult for them to work together – the raw tran- scripts generated by ASR contains no segmenta- tion (see Table 1 for an example), while MT ex- pects segmented sentences as input. Online sentence segmentation smoothly bridges the gap between ASR and MT through segment- ing the transcripts generated by ASR engines into sentences in real time. As a matter of fact, the task is non-trivial. The example presented in Table 1 is extracted from a TED talk 2, which is used in the experiments of this paper. Readers may find the raw sequence of words difficult to read. How- ever, the readability is greatly improved once it is segmented as follows, • even cats were watching this video • cats were watching other cats watch this video • but what ’s important here is the creativity that it inspired amongst this techie geeky in- ternet culture • there were remixes • someone made an old timey version 2https://www.ted.com/ Proceedings of MT Summit XVII, volume 1 Dublin, Aug. 19-23, 2019 | p. 1 • …

Content truncated to 500 words