Assumptions, expectations and outliers in post-editing

Assumptions, Expectations and Outliers in Post-Editing Laura Casanellas Welocalize / Dublin, Ireland laura.casanellas@welocalize.com Lena Marg Welocalize / London lena.marg@welocalize.com Abstract As a multilingual vendor, we have access to machine translation (MT) scoring and other evaluation data on a wide range of language combinations and content types; we also have experience with different MT systems in production. Our daily work involves the collaboration with a wide spectrum of translation partners, from very MT-savvy to novices in this area. Being exposed to MT in such a var- ied and large-scale setup, we would like to share some of our insights into as- sumptions, expectations and outliers ob- served with regard to MT quality, productivity and suitability with a partic- ular focus on the challenges that (indi- vidual) post-editor behavior presents in this context. Our observations are based on data correlations carried out at the end of 2013 from a database that contains all evaluation data produced during this year, as well as recent surveys with some of our very MT-savvy translation partners for deeper, locale-specific insights. 1 Introduction In our company machine translation (MT) is typically integrated in the translation workflow as a productivity tool complementing translation memories, glossaries etc., with translators carrying out the required levels of post-editing. Content is translated into a multitude of languages (mostly from English) and MT is currently being used in production on a wide range of content types, from technical communication, user interface and corporate © 2014 by Welocalize. communication to user-generated contents. Additionally, we do not work with a specific MT system, but rather a variety of MT systems are evaluated and used – based on our own or our client’s recommendations. At the end of 2013, we created a comprehensive database of results from automatic and human scorings of MT output as well as results from productivity tests obtained in that year, covering all these variables (locales, content types, MT systems). Our productivity test shows the potential productivity gains obtained by moving from the task of translating to post-editing. While the analysis and correlations drawn from this database confirmed certain assumptions, it allowed us to reassess expectations and also provided insights into outliers. In this abstract and our presentation, we would like to discuss these assumptions, expectations and outliers, benefitting from the wide range of variables used in the company. In this context, we want to draw attention to individual translator behavior, which might need to be considered more strongly when assessing MT output quality and usability. 2 The Database The database mentioned above was created with all available data related to MT evaluation from 2013. The timeframe was delimited to one year. Objectives for creating the database were mul- tiple, but a key aim was to see if a correlation of currently available, internal data would help us make productivity predictions on future MT post-editing effort with the metrics currently in use in the company. The categories included in the 2013 database are: client name, content type, locale, translation partner carrying out …

Content truncated to 500 words