PosEdiOn: Post-Editing Assessment in PythOn

PosEdiOn: Post-Editing Assessment in PythOn Antoni Oliver Universitat Oberta de Catalunya aoliverg@uoc.edu Sergi Alvarez Universitat Pompeu Fabra salvarezvid@uoc.edu Toni Badia Universitat Pompeu Fabra toni.badia@upf.edu Abstract There is currently an extended use of post- editing of machine translation (PEMT) in the translation industry. This is due to the increase in the demand of translation and to the significant improvements in quality achieved in recent years. PEMT has been included as part of the translation work- flow because it increases translators’ pro- ductivity and it also reduces costs. Al- though effective post-editing requires suf- ficiently high quality MT output, usual au- tomatic metrics do not always correlate with post-editing effort. We describe a standalone tool designed both for indus- try and research that has two main pur- poses: to collect sentence-level informa- tion from the post-editing process (e.g. post-editing time and keystrokes) and to visually present multiple evaluation scores so they can be easily interpreted by a user. 1 Introduction Post-editing of machine translation (PEMT) is a very common practice in the translation indus- try. It has been included as part of the translation workflow because it increases productivity when compared with human translation (Aranberri et al., 2014) and reduces costs (Guerberof, 2009) with- out having a negative impact on quality (Plitt and Masselot, 2010). Post-editors “edit, modify and/or correct pre-translated text that has been processed by an MT system from a source language into (a) target language(s)” (Allen, 2003, p. 296). c⃝ 2020 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. In the last few years, both research and indus- try have become very interested in neural machine translation (NMT) because it has produced very successful results in terms of quality, for exam- ple in WMT 2017 (Bojar et al., 2017), WMT 2018 (Bojar et al., 2018) and WMT 2019 (Barrault et al., 2019). Given the overall performance of NMT, it is necessary to study all the potential this approach can offer to post-editing. One of the main prob- lems is that automatic scores give a general idea of the MT output quality but do not always corre- late with post-editing effort (Koponen, 2016; Shte- rionov et al., 2018). Many professional translators state that if the quality of the MT output is not good enough, they delete the remaining segments and translate everything from scratch (Parra Escart´ın and Arcedillo, 2015). One of the main goals both of industry and re- search is to establish a correlation between the quality measurements of the MT output and trans- lators’ performance. Research is especially fo- cused on the effort this activity entails, mainly taking into account the temporal, technical, and cognitive effort (Krings, 2001). The use of tools that can log these three dimensions becomes a paramount challenge for research. Professional translators usually use commercial products to translate and post-edit. In the 2018 Language Industry Survey1 conducted by EUATC, Elia, FIT Europe, GALA and LINDWeb, SDL Tra- dos2 was the most used product with …

Content truncated to 500 words