VUS at IWSLT 2021: A Finetuned Pipeline for Offline Speech Translation

Proceedings of the 18th International Conference on Spoken Language Translation, pages 120–124 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 120 VUS at IWSLT 2021: A Finetuned Pipeline for Offline Speech Translation Yong Rae Jo Voithru Inc. Jungyoon Choi Voithru Inc. Young Ki Moon Voithru Inc. Jihyung Moon Upstage {yongrae.jo, minji.jung, jungyoon.choi}@voithru.com ykmoon0814@gmail.com, jihyung.moon@upstage.ai tsatsuki@snu.ac.kr Minji Jung Voithru Inc. Won Ik Cho Seoul National University Abstract In this technical report, we describe the fine-tuned1 ASR-MT pipeline used for the IWSLT shared task. We remove less useful speech samples by checking WER with an ASR model, and further train a wav2vec and Transformers-based ASR module based on the filtered data. In addition, we cleanse the er- rata that can interfere with the machine trans- lation process and use it for Transformer-based MT module training. Finally, in the actual inference phase, we use a sentence boundary detection model trained with constrained data to properly merge fragment ASR outputs into full sentences. The merged sentences are post- processed using part of speech. The final re- sult is yielded by the trained MT module. The performance using the dev set displays BLEU 20.37, and this model records the performance of BLEU 20.9 with the test set. 1 Introduction Offline speech translation is a task that infers the text of a target language by using speech as in- put. A pipeline system is used as a representa- tive method, which converts source speech into the source text via automatic speech recognition and machine translates it. Recently, many speech corpora have been disclosed, and studies are be- ing conducted on an end-to-end method, namely directly decoding speech input into the text of a target language (B´erard et al., 2016, 2018). In this IWSLT shared offline task, we implement an English-German speech translation system in a pipeline format. The advantage of pipeline architec- ture is that it can explain whether the given speech translation is challenging in view of the acoustic domain or the translation perspective, considering 1We use ‘fine-tuned’ to describe that our approach is not fully end-to-end but incorporates a well-organized set of strate- gies to reach better performance. It does not denote the wav2vec-transformer ASR module either. the whole process of converting source speech to the target text. This makes it easier for us to dis- cern difficult or erroneous parts in speech and text processing. In general, a limitation of a pipeline system com- pared to an end-to-end system is that the quality of the final result is largely influenced by the in- termediate text representation, which is usually obtained in an explicit format (Liu et al., 2020). Therefore, we primarily remove training samples that can lower the ASR performance, following the method used in Potapczyk and Przybysz (2020). Thereafter, based on the trained ASR module, the output of test speech samples is transformed into the text and fed to the machine translation system to produce a final output. In this process, we con- duct post-processing to obtain an accurate …

Content truncated to 500 words