Inverted Projection for Robust Speech Translation

Proceedings of the 18th International Conference on Spoken Language Translation, pages 236–244 Bangkok, Thailand (Online), August 5–6, 2021. ©2021 Association for Computational Linguistics 236 Inverted Projection for Robust Speech Translation Dirk Padfield∗ Google Research padfield@google.com Colin Cherry∗ Google Research colincherry@google.com Abstract Traditional translation systems trained on writ- ten documents perform well for text-based translation but not as well for speech-based ap- plications. We aim to adapt translation models to speech by introducing actual lexical errors from ASR and segmentation errors from au- tomatic punctuation into our translation train- ing data. We introduce an inverted projec- tion approach that projects automatically de- tected system segments onto human transcripts and then re-segments the gold translations to align with the projected human transcripts. We demonstrate that this overcomes the train-test mismatch present in other training approaches. The new projection approach achieves gains of over 1 BLEU point over a baseline that is exposed to the human transcripts and segmen- tations, and these gains hold for both IWSLT data and YouTube data. 1 Introduction Speech translation is an important field that be- comes more relevant with every improvement to its component technologies of automatic speech recognition (ASR) and machine translation (MT). It enables exciting applications like live machine interpretation (Cho and Esipova, 2016; Ma et al., 2019) and automatic foreign-language subtitling for video content (Karakanta et al., 2020). However, translation of speech presents unique challenges compared to text translation. Traditional text translation systems are often trained with clean, well-structured text consisting of (source language, target language) sentence pairs gathered from text documents. This works well for translating written text, but for cascaded systems composed of speech → automatic transcription → automatic transla- tion, errors from ASR and automatic punctuation are amplified as they pass through the translation ∗equal contribution system. Such systems suffer from three issues: 1) spoken language structure is different from written language structure and can include aspects like dis- fluencies and partial sentences, 2) ASR systems are not perfect and introduce errors in the stage from speech to source transcript, and 3) mistakes from automatic punctuation systems can lead to unnat- ural sentence segments and boundaries (Makhija et al., 2019; Nguyen et al., 2019; Wang et al., 2019). These problems can lead to poor translations and pose unique challenges for MT that are not readily addressed by current methods. In this work, we set out to make MT robust to the second and third issues in particular. We have developed an approach to train transla- tion models that are robust to transcription errors and punctuation errors, by introducing errors from actual ASR and automatic punctuation systems into the source side of our MT training data. This is sim- ilar in spirit to the method of Li et al. (2021), which introduces artificial sentence boundary errors into the training bitext. However, instead of artificial boundaries, our segmentation approach uses actual boundaries generated by the automatic punctuation system, which required the development of our in- verted projection technique, and we also include errors …

Content truncated to 500 words