Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain

Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain Samuel Läubli 1,3 Chantal Amrhein 1,3 Patrick Düggelin 1 Beatriz Gonzalez 2 Alena Zwahlen 1 Martin Volk 1,3 1 TextShuttle AG 2 Migros Bank AG 3 Institute of Computational Linguistics, University of Zurich Abstract Neural machine translation (NMT) has set new quality standards in automatic transla- tion, yet its effect on post-editing produc- tivity is still pending thorough investiga- tion. We empirically test how the inclusion of NMT, in addition to domain-specific translation memories and termbases, im- pacts speed and quality in professional translation of financial texts. We find that even with language pairs that have re- ceived little attention in research settings and small amounts of in-domain data for system adaptation, NMT post-editing al- lows for substantial time savings and leads to equal or slightly better quality. 1 Introduction The use of neural networks for sequence transduc- tion (Kalchbrenner and Blunsom, 2013; Sutskever et al., 2014; Bahdanau et al., 2015) has led to as- tounding progress in the field of machine trans- lation (MT), establishing a new level of quality in applications such as automatic news translation (Sennrich et al., 2016b; Hassan et al., 2018). Nev- ertheless, the creation of publication-grade trans- lations still requires human involvement (Läubli et al., 2018; Toral et al., 2018a), and previous work has explored human–machine collaboration in the form of post-editing, where human experts revise machine translated text where necessary. Empirical investigations of post-editing produc- tivity with NMT are still scarce, especially for lan- guage combinations that do not include English as c⃝ 2019 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. either the source or target language. In this paper, we describe and discuss the results of a productiv- ity test of domain-adapted NMT with the in-house translation team of Migros Bank, a mid-sized fi- nancial institution based in Zurich, Switzerland. We evaluate the use of NMT under typical work- ing conditions, focussing on two rarely explored language pairs—German (DE) to French (FR) and Italian (IT)—and texts from a specialised domain: banking and finance. We show that using NMT in combination with translation memories (TMs) and terminology databases (TBs) enables professional translators to work faster with no loss (or slight gains) in quality, even with limited in-domain data for system adaptation. 2 Background Early assessments of post-editing productivity were focussed on technical texts. While a study by Krings (1994) with user manuals for techni- cal appliances and rule-based MT found mixed re- sults, interest in post-editing grew with the advent of statistical MT (SMT), which enabled time sav- ings of up to 40 % in film subtitling (Volk, 2008; de Sousa et al., 2011) and software localisation (Plitt and Masselot, 2010). Subsequent work con- cluded that significant time savings can also be achieved in more complex domains such as legal (Federico et al., 2012) or marketing texts (Läubli et al., 2013). Many productivity …

Content truncated to 500 words