Translator2Vec: Understanding and Representing Human Post-Editors

Translator2Vec: Understanding and Representing Human Post-Editors Ant´onio G´ois Unbabel Lisbon, Portugal antonio.gois@unbabel.com Andr´e F. T. Martins Unbabel & Instituto de Telecomunicac¸˜oes Lisbon, Portugal andre.martins@unbabel.com Abstract The combination of machines and humans for translation is effective, with many stud- ies showing productivity gains when hu- mans post-edit machine-translated output instead of translating from scratch. To take full advantage of this combination, we need a fine-grained understanding of how human translators work, and which post-editing styles are more effective than others. In this paper, we release and an- alyze a new dataset with document-level post-editing action sequences, including edit operations from keystrokes, mouse ac- tions, and waiting times. Our dataset com- prises 66,268 full document sessions post- edited by 332 humans, the largest of the kind released to date. We show that ac- tion sequences are informative enough to identify post-editors accurately, compared to baselines that only look at the initial and final text. We build on this to learn and vi- sualize continuous representations of post- editors, and we show that these representa- tions improve the downstream task of pre- dicting post-editing time. 1 Introduction Computer-aided translation platforms for interac- tive translation and post-editing are now com- monly used in professional translation services (Alabau et al., 2014; Federico et al., 2014; Green et al., 2014; Denkowski, 2015; Hokamp, 2018; Sin-wai, 2014; Kenny, 2011). With the in- creasing quality of machine translation (Bahdanau c⃝ 2019 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. et al., 2014; Gehring et al., 2017; Vaswani et al., 2017), the translation industry is going through a transformation, progressively shifting gears from “computer-aided” (where MT is used as an in- strument to help professional translators) towards human-aided translation, where there is a hu- man in the loop who only intervenes when needed to ensure final quality, and whose productivity is to be optimized. A deep, data-driven understand- ing of the human post-editing process is key to achieve the best trade-offs in translation efficiency and quality. What makes a “good” post-editor? What kind of behaviour shall an interface pro- mote? There is a string of prior work that relates the difficulty of translating text with the cognitive load of human translators and post-editors, based on in- dicators such as editing times, pauses, keystroke logs, and eye tracking (O’Brien, 2006; Doherty et al., 2010; Lacruz et al., 2012; Balling and Carl, 2014, see also §6). Most of these studies, however, have been performed in controlled environments on a very small scale, with a limited number of professional translators and only a few sessions. A direct use of human activity data for understand- ing and representing human post-editors, towards improving their productivity, is still missing, ar- guably due to the lack of large-scale data. Under- standing how human post-editors work could open the door to the design of better interfaces, smarter allocation of human translators to content, and au- tomatic post-editing. In this paper, we study the behaviour of hu- man …

Content truncated to 500 words