Error Annotation in Post-Editing Machine Translation: Investigating the Impact of Text-to-Speech Technology

Error Annotation in Post-Editing Machine Translation: Investigating the Impact of Text-to-Speech Technology Justus Brockmann Centre for Translation Studies University of Vienna justus.brockmann @univie.ac.at Claudia Wiesinger Centre for Translation Studies University of Vienna claudia.wiesinger @univie.ac.at Dragoș Ciobanu Centre for Translation Studies University of Vienna dragos.ioan.ciobanu @univie.ac.at Abstract As post-editing of machine translation (PEMT) is becoming one of the most dominant services offered by the language services industry (LSI), efforts are being made to support the provision of this service with additional technologies. We present text-to-speech (T2S) as a potential attention-raising technology for post- editors. Our study was conducted with university students and included both PEMT and MT error annotation of a creative text with and without T2S. Focusing on the error annotation data, our analysis finds that participants under- annotated fewer MT errors in the T2S condition compared to the silent condition. At the same time, more over-annotation was recorded. Finally, annotation performance corresponded to participants’ attitudes towards using T2S. 1 Introduction With machine translation (MT) adoption and the provision of post-editing machine translation (PEMT) services on the rise, Translation Process Research (TPR) has been questioning whether the ways in which PEMT is currently being carried out (in dedicated PEMT tools, in simple word processing software, or in computer-assisted translation (CAT) tools/translation environment tools (TEnT), and with or without the use of additional technologies) optimally support post- editors, both from a process- and a product- oriented point of view (Moorkens and O’Brien, 2017). As the technological possibilities are © 2022 Brockmann, J., Wiesinger, C., & Ciobanu, D. This article is licensed under a Creative Commons 3.0 license, no derivative works, attribution, CC-BY-ND. growing, there is an uptake of speech tools such as automatic speech recognition (ASR; speech-to- text) by professional translators (ELIA et al., 2022), and this practice has become one of the focal points of TPR (Dragsted, Mees and Hansen, 2011; Ciobanu, 2014, 2016; Mesa-Lao, 2014; Zapata, Castilho and Moorkens, 2017; Liyanapathirana, 2021). While the use of automatic speech synthesis (text-to-speech; T2S) has received comparatively little attention both from the language services industry (LSI) and the research community, translators and revisers are known to read aloud translations during (self-)revision (Allain, 2010; Ciobanu, 2016; Scocchera, 2017). This intuitively perceived benefit of aurally processing a text points to the potential of T2S as an attention- raising technology that may also help post-editors identify subtle neural machine translation (NMT) errors. The practice of PEMT remains a particular challenge for Translation Studies students, despite the transition from statistical MT (SMT) to NMT which has reduced the absolute number of errors to be corrected in the raw MT output (Yamada, 2019). Moreover, the phenomena of over- and under-editing continue to preoccupy both academia and the LSI (Nitzke and Gros, 2020). We share the view that students need to be exposed to a variety of translation tools early and often, and we believe that introducing them to additional technologies such as T2S will prove beneficial for honing the skills needed to succeed as future post-editors. Rather …

Content truncated to 500 words