Efforts on Machine Learning over Human-mediated Translation Edit Rate

Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 302–306, Baltimore, Maryland USA, June 26–27, 2014. c⃝2014 Association for Computational Linguistics Efforts on Machine Learning over Human-mediated Translation Edit Rate Eleftherios Avramidis German Research Center for Artificial Intelligence (DFKI) Language Technology Lab Alt Moabit 91c, 10559 Berlin, Germany eleftherios.avramidis@dfki.de Abstract In this paper we describe experiments on predicting HTER, as part of our submis- sion in the Shared Task on Quality Esti- mation, in the frame of the 9th Workshop on Statistical Machine Translation. In our experiment we check whether it is possi- ble to achieve better HTER prediction by training four individual regression models for each one of the edit types (deletions, insertions, substitutions, shifts), however no improvements were yielded. We also had no improvements when investigat- ing the possibility of adding more data from other non-minimally post-edited and freely translated datasets. Best HTER pre- diction was achieved by adding dedupli- cated WMT13 data and additional features such as (a) rule-based language correc- tions (language tool) (b) PCFG parsing statistics and count of tree labels (c) posi- tion statistics of parsing labels (d) position statistics of tri-grams with low probability. 1 Introduction As Machine Translation (MT) gets integrated into regular translation workflows, its use as base for post-editing is radically increased. As a result, there is a great demand for methods that can auto- matically assess the MT outcome and ensure that it is useful for the translator and can lead to more productive translation work. Although many agree that the quality of the MT output itself is not adequate for the profes- sional standards, there has not yet been a widely- accepted way to measure its quality on par with human translations. One such metric, the Hu- man Translation Edit Rate (HTER) (Snover et al., 2006), is the focus of the current submission. HTER is highly relevant to the need of adapting MT to the needs of translators, as it aims to mea- sure how far it is from an acceptable equivalent translation done by humans. HTER is used here in the frame of Quality Es- timation, i.e. having the goal of being able to pre- dict the post-editing effort in a real case environ- ment, right before the translation is given to the user, without real access to the correct translation. For this purpose the text of the source and the pro- duced translation is analyzed by automatic tools in order to infer indications (numerical features) that may be relevant to the quality of the transla- tion. These features are used in a statistical model whose parameters are estimated with common su- pervised Machine Learning techniques. This work presents an extensive search over var- ious set-ups and parameters for such techniques, aiming to build a model that better predicts HTER over the data of the Shared Task of the 9th Work- shop on Statistical Machine Translation. 2 New approaches being tested 2.1 Break HTER apart HTER is a complex metric, in the sense that it is …

Content truncated to 500 words