Direct Segmentation Models for Streaming Speech Translation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2599–2611, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2599 Direct Segmentation Models for Streaming Speech Translation Javier Iranzo-S´anchez, Adri`a Gim´enez Pastor, Joan Albert Silvestre-Cerd`a, Pau Baquero-Arnal, Jorge Civera, Alfons Juan Machine Learning and Language Processing (MLLP) research group Valencian Research Institute for Artificial Intelligence (VRAIN) Universitat Politcnica de Valncia, Spain {jairsan,adgipas,jsilvestre,pabaar,jorcisai,ajuanci}@vrain.upv.es Abstract The cascade approach to Speech Translation (ST) is based on a pipeline that concatenates an Automatic Speech Recognition (ASR) sys- tem followed by a Machine Translation (MT) system. These systems are usually connected by a segmenter that splits the ASR output into, hopefully, semantically self-contained chunks to be fed into the MT system. This is specially challenging in the case of streaming ST, where latency requirements must also be taken into account. This work proposes novel segmenta- tion models for streaming ST that incorporate not only textual, but also acoustic information to decide when the ASR output is split into a chunk. An extensive and thorough experi- mental setup is carried out on the Europarl-ST dataset to prove the contribution of acoustic in- formation to the performance of the segmenta- tion model in terms of BLEU score in a stream- ing ST scenario. Finally, comparative results with previous work also show the superiority of the segmentation models proposed in this work. 1 Introduction ST is a field that is very closely aligned with ASR and MT, as it is their natural evolution to com- bine the advances in both areas. Thus, the goal is to obtain the translation of an utterance that has been spoken in a different language, without neces- sarily requiring the intermediate transcription. At the same time, it is desirable to have high qual- ity translations without compromising the speed of the system. Although research into ST started in the nineties (Waibel et al., 1991), the field did not really take off until significant breakthroughs were achieved in ASR (Chan et al., 2016; Irie et al., 2019; Park et al., 2019; Jorge et al., 2020) and MT (Bahdanau et al., 2015; Sennrich et al., 2016b,a; Vaswani et al., 2017), mainly due to the introduction of deep neural networks (NN). Thanks to this, the field has recently attracted significant amounts of attention from both the research and industry communities, as the field is now mature enough that it has tangible and well-performing applications (Ma et al., 2019; Jia et al., 2019). Currently, there are two main approaches to ST: cascade and end-to-end models. The goal of the end-to-end approach is to train a single system that is able to carry out the the entire translation process (Weiss et al., 2017; Berard et al., 2018; Gangi et al., 2019). This has only recently been possible thanks to advances in neural modeling. Due to a lack of ST data, different techniques such as pre-training and data augmentation (Bahar et al., 2019; Pino et al., 2019) have been used in order to alleviate this lack …

Content truncated to 500 words