Correlating Translation Product and Translation Process Data of Professional and Student Translators

Correlating Translation Product and Translation Process Data of Professional and Student Translators Michael Carl and Matthias Buch-Kromann Dept. of International Languages Studies & Computational Linguistics, Copenhagen Business School, 2000 Frederiksberg, Denmark {mc.isv;mbk.isv}@cbs.dk Abstract The paper presents an exploratory study of the translation processes for 12 student and 12 professional translators. We re- late properties of the translators’ process data (eye movements and keystrokes) with the quality of the produced translations, using BLEU scores and human evalua- tion scores for fluency and accuracy to as- sess translation quality. We also investi- gate how BLEU scores correlate with hu- man scores, and how BLEU scores de- pend on the number of reference transla- tions. We segment the translation process into skimming, drafting and post-editing phases, and show that the translation be- havior of student and professional transla- tors differ with respect to how they use the translation phases. We also show that stu- dents and professionals differ mainly with respect to produced translation fluency. 1 Introduction Although machine translation quality has in- creased over the past years, current state-of-the- art general-purpose MT rarely meets high quality standards without human intervention. A number of tools for translation assistance have been pro- posed, such as Translation Memories, MT post- editing tools, and interactive MT. While TMs are widely adopted in the translation industry, they do not include all the possible translation aids that can be provided by a computer today, and the full po- tential for the utilization of MT in human transla- tion has not been reached yet. However, to provide better MT-based support for human translators, we c⃝ 2010 European Association for Machine Translation. need a better understanding of human translation processes in different groups of human translators, the bottlenecks experienced by these translators, and how the bottlenecks can be mitigated by au- tomated assistance. In this paper, we are particularly interested in the differences between professional translators and student translators. For instance, do professionals produce measurably better (or different) transla- tions than students? Do the two groups differ with respect to their working styles? More generally, are there kinds of automated translation assistance that are most helpful in a crowd translation context, and others that are better targeted towards profes- sional translators, or do we need the same tools in both cases? To approach these questions, we analyze the user activity data (eye movements and keystrokes) of 12 student and 12 professional translators trans- lating two small English texts into Danish. The hu- man translations, as well as a machine translation from produced by Google Translate, are evaluated and compared, both automatically with BLEU and manually with human scores for fluency and accu- racy. We also analyze the translation process data and correlate them with the translation quality. In Section 2, we investigate and quantify dif- ferent translation phases (skimming, drafting, re- vision) of student and professional translators and analyze differences in their translation behavior. In Section 3, we look at the quality of the produced translations in terms of BLEU …

Content truncated to 500 words