How do LSPs compute MT discounts? Presenting a company’s pipeline and its use

How do LSPs compute MT discounts? Presenting a company’s pipeline and its use Randy Scansani Acolad Rimini, Italy rscansani@acolad.com Lamis Mhedhbi Acolad Paris, France lmhedhbi@acolad.com Abstract In this paper we present a pipeline devel- oped at Acolad to test a Machine Trans- lation (MT) engine and compute the dis- count to be applied when its output is used in production. Our pipeline includes three main steps where quality and productivity are measured through automatic metrics, manual evaluation, and by keeping track of editing and temporal effort during a post-editing task. Thanks to this approach, it is possible to evaluate the output qual- ity and compute an engine-specific dis- count. Our test pipeline tackles the com- plexity of transforming productivity mea- surements into discounts by comparing the outcome of each of the above-mentioned steps to an estimate of the average pro- ductivity of translation from scratch. The discount is obtained by subtracting the re- sulting coefficient from the per-word rate. After a description of the pipeline, the pa- per presents its application on four en- gines, discussing its results and showing that our method to estimate post-editing effort through manual evaluation seems to capture the actual productivity. The pipeline relies heavily on the work of pro- fessional post-editors, with the aim of cre- ating a mutually beneficial cooperation be- tween users and developers. 1 Introduction Over the last few years, the number of compa- nies starting to integrate machine translation (MT) c⃝ 2020 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. in their workflow has increased dramatically. In 2018, for the first time, more than half of the com- panies and individual professionals taking part in the Language Industry Survey stated that they used MT (Elia et al., 2018). The same survey repeated in 2019 showed that MT was one of the highest priorities for companies, with 51% of them willing to increase its use and 62% stating that they were planning investments on MT (Elia et al., 2019). At the same time, all categories involved in the 2019 Language Industry Survey mentioned price pressure as the main negative trend, with MT and post-editing identified as one of the major causes. In the previous year, one of the main concerns of the language industry components were the “tech- nological advances that are not initiated or con- trolled by the respondents” (Elia et al., 2018, p. 31). These surveys point out that individuals and companies are embracing a technology that they themselves see as a threat to the sustainability of the translation industry. This seems to suggest that there might be a strong disagreement regarding the price models to be adopted when MT is included in the workflow. One of the priorities of MT users and developers should therefore be the creation of shared models and/or methods to measure qual- ity and productivity with a view to computing dis- counts to be applied when MT is used. However, to the best of …

Content truncated to 500 words