Beyond Sentence-Level End-to-End Speech Translation: Context Helps

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2566–2578 August 1–6, 2021. ©2021 Association for Computational Linguistics 2566 Beyond Sentence-Level End-to-End Speech Translation: Context Helps Biao Zhang1 Ivan Titov1,2 Barry Haddow1 Rico Sennrich3,1 1School of Informatics, University of Edinburgh 2ILLC, University of Amsterdam 3Department of Computational Linguistics, University of Zurich B.Zhang@ed.ac.uk, {ititov,bhaddow}@inf.ed.ac.uk, sennrich@cl.uzh.ch Abstract Document-level contextual information has shown benefits to text-based machine transla- tion, but whether and how context helps end- to-end (E2E) speech translation (ST) is still under-studied. We fill this gap through exten- sive experiments using a simple concatenation- based context-aware ST model, paired with adaptive feature selection on speech encodings for computational efficiency. We investigate several decoding approaches, and introduce in- model ensemble decoding which jointly per- forms document- and sentence-level transla- tion using the same model. Our results on the MuST-C benchmark with Transformer demon- strate the effectiveness of context to E2E ST. Compared to sentence-level ST, context-aware ST obtains better translation quality (+0.18- 2.61 BLEU), improves pronoun and homo- phone translation, shows better robustness to (artificial) audio segmentation errors, and re- duces latency and flicker to deliver higher qual- ity for simultaneous translation.1 1 Introduction Document-level context often offers extra informa- tive clues that could improve the understanding of individual sentences. Such clues have been proven effective for textual machine translation (MT), par- ticularly in handling translation errors specific to discourse phenomena, such as inaccurate corefer- ence of pronouns (Guillou, 2016) and mistransla- tion of ambiguous words (Rios et al., 2017). Be- sides, ensuring consistency in translation is virtu- ally impossible without document-level context as well (Voita et al., 2019). Analogous to MT, speech translation (ST) also suffers from these translation issues, and super-sentential context could in fact be more valuable to ST because 1) homophones 1Source code is available at https://github.com/ bzhangGo/zero. Figure 1: Overview of the concatenation-based context- aware ST. yn denotes the n-th target sentence in a document; xn denotes the speech encodings extracted from the n-th audio segment. We use dashed gray box to indicate the concatenation operation. “<s>”: sentence separator symbol. and acoustic noise bring additional ambiguity to ST, and 2) a common use case in ST is simulta- neous translation, where the system has to output translations of sentence fragments, and may have to predict future input to account for word order differences between the source and target language (Grissom II et al., 2014). Both for ambiguity from the acoustic signal, and operating on small sentence fragments, we hypothesize that access to extra con- text2 will be beneficial. Although recent studies on ST have achieved promising results with end-to-end (E2E) mod- els (Anastasopoulos and Chiang, 2018; Di Gangi et al., 2019; Zhang et al., 2020a; Wang et al., 2020; Dong et al., 2020), nevertheless, they mainly focus on sentence-level translation. One practical chal- lenge when scaling up sentence-level E2E ST to the document-level is the encoding of very long audio segments, which can …

Content truncated to 500 words