Who Are We Talking About? Handling Person Names in Speech Translation

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 62 - 73 May 26-27, 2022 c⃝2022 Association for Computational Linguistics Who Are We Talking About? Handling Person Names in Speech Translation Marco Gaido1,2, Matteo Negri1 and Marco Turchi1 1Fondazione Bruno Kessler 2University of Trento Trento, Italy {mgaido,negri,turchi}@fbk.eu Abstract Recent work has shown that systems for speech translation (ST) – similarly to automatic speech recognition (ASR) – poorly handle person names. This shortcoming does not only lead to errors that can seriously distort the meaning of the input, but also hinders the adoption of such systems in application scenarios (like computer- assisted interpreting) where the translation of named entities, like person names, is crucial. In this paper, we first analyse the outputs of ASR/ST systems to identify the reasons of fail- ures in person name transcription/translation. Besides the frequency in the training data, we pinpoint the nationality of the referred person as a key factor. We then mitigate the problem by creating multilingual models, and further improve our ST systems by forcing them to jointly generate transcripts and translations, pri- oritising the former over the latter. Overall, our solutions result in a relative improvement in token-level person name accuracy by 47.8% on average for three language pairs (en→es,fr,it). 1 Introduction Automatic speech translation (ST) is the task of generating the textual translation of utterances. Re- search on ST (Anastasopoulos et al., 2021; Ben- tivogli et al., 2021) has so far focused on compar- ing the cascade (a pipeline of an automatic speech recognition – ASR – and a machine translation – MT – model) and direct paradigms (Bérard et al., 2016; Weiss et al., 2017), or on improving either of them in terms of overall quality. Quality is usually measured with automatic metrics such as BLEU (Papineni et al., 2002) and TER (Snover et al., 2006), possibly corroborated by manual analyses. These metrics – as well as neural-based ones like COMET (Rei et al., 2020) – are relatively insensi- tive to errors on named entities (NEs) and numbers (Amrhein and Sennrich, 2022), which instead are of paramount importance for human readers (Xie et al., 2022). As such, the blind pursue of higher scores can lead to systems biased toward the met- rics and not targeted on real users. In addition, there are cases in which users are in- terested only in NEs. For instance, interpreters easily craft more fluent and intelligible transla- tions than machines (Fantinuoli and Prandi, 2021), but during simultaneous sessions suffer from a high cognitive workload (Prandi, 2018; Desmet et al., 2018), to which NEs and specific termi- nology significantly contribute (Jones, 1998; Gile, 2009; Prandi, 2018; Desmet et al., 2018). Indeed, these elements i) are hard to remember (Liu et al., 2004), ii) can be unknown to interpreters and diffi- cult to recognize (Griffin and Bock, 1998), and iii) differently from other types of words, usu- ally have one or few correct translations. For this reason, modern computer-assisted interpret- ing (CAI – Fantinuoli …

Content truncated to 500 words