Determining translators’ perception, productivity and post-editing effort when using SMT and NMT systems

Determining translators’ perception, productivity and post-editing effort when using SMT and NMT systems. Ariana López Pereira Universitat Autònoma de Barcelona arianalopezp@gmail.com Abstract Thanks to the great progress seen in the machine translation (MT) field in recent years, the use and perception of MT by translators need to be revisited. The main objective of this paper is to determine the perception, productivity and the post- editing effort (in terms of time and num- ber of editings) of six translators when using Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) systems. This presentation is fo- cused on how translators perceive these two systems in order to know which one they prefer and what type of errors and problems present each system, as well as how translators solve these issues. These tests will be performed with the Dynamic Quality Framework (DQF) tools (quick comparison and productivity tasks) using Google Neural Machine Translation and Microsoft Translator (SMT) APIs in two different English into Spanish texts, an instruction manual and a marketing webpage. Results showed that translators considerably prefer NMT over SMT. Moreover, NMT is more adequate and fluent than SMT. 1 Introduction Machine Translation (MT) is nowadays one of the most useful resources for translators and the translation industry. Post-editing has become a usual practice within companies (Torres Hostench et al., 2016). With the great progress seen in NMT (Castilho et al., 2017), there are still some problems to overcome when using it, © 2018 The authors. This article is licensed under a Creative Commons 3.0 license, no derivative works, at- tribution, CC-BY-ND. especially regarding terminology issues. Despite these innovations, SMT systems are still very popular. Hence, it is important to discover the differences between the two systems in order to use them properly. 2 Aim of this proposal The aim of this paper is to determine the translators’ perception when using SMT and NMT, as well as to observe the differences when using SMT and NMT based on the topic of the source text. The research questions addressed will be: • Do translators prefer SMT or NMT? • Which issues present the use of SMT and which ones NMT? Does the SMT present more accurate results? Is the NMT more fluid? • Are these issues different based on the topic of the text (marketing and user documentation source texts)? • How do the translators post-edit these is- sues? Results showed that the translators preferred NMT, which was more fluent and adequate than SMT. NMT was both more adequate and fluid, both for the instruction manual and the marketing texts. SMT presented best results in the market- ing test, compared to the user documentation test. References Castilho, S., Moorkens, J., Gaspari, F., Calixto, I., Tinsley, J., & Way, A. (2017). Is Neural Machine Translation the New State of the Art? The Prague Bulletin of Mathematical Linguistics, 108(1). https://doi.org/10.1515/pralin-2017-0013 Torres Hostench, O.; Cid-Leal, P. and Presas, M. (coord.) (2016). El uso de traducción automática y posedición en las empresas de servicios lingüísti- cos españolas: …

Content truncated to 500 words