UHH Submission to the WMT17 Quality Estimation Shared Task

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 556–561 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics UHH Submission to the WMT17 Quality Estimation Shared Task Melania Duma and Wolfgang Menzel University of Hamburg Natural Language Systems Division {duma, menzel}@informatik.uni-hamburg.de Abstract The field of Quality Estimation (QE) has the goal to provide automatic methods for the evaluation of Machine Translation (MT), that do not require reference transla- tions in their computation. We present our submission to the sentence level WMT17 Quality Estimation Shared Task. It com- bines tree and sequence kernels for pre- dicting the post-editing effort of the tar- get sentence. The kernels exploit both the source and target sentences, but also a back-translation of the candidate transla- tion. The evaluation results show that the kernel approach combined with the base- line features brings substantial improve- ment over the baseline system. 1 Introduction The evaluation of Machine Translation (MT) out- put is a sub-field of MT research that has experi- enced a great amount of interest in the past years. The process of MT evaluation involves three fac- tors: an input segment in a source language, the candidate translation (also known as target sen- tence) which represents the output of a MT sys- tem when translating from the source language to the target language and a reference translation in the target language. The assessment of MT qual- ity can be divided into two categories depending on whether it requires the presence of a reference translation or not. The reference-based evaluation scores the candidate translation by comparing it to the reference translation. On the other hand, the reference-free evalua- tion, also known as quality estimation (QE), pre- dicts the quality of a candidate translation based solely on the information contained in the source and target sentences. QE can be performed at different levels of granularity: word, sentence or phrase and it involves classifying, ranking or pre- dicting scores for the candidate translations. sentence-level QE system is conventionally con-A structed based on a set of features encoding the in- formation contained in the source and target sen- tences, which are used for learning a prediction model. The features employed for this task can be of different types, like surface features, language model features or linguistic features. The positive influence of syntactic features on the performance of QE systems has been extensively studied, in- cluding in Rubino et al. (2012), Avramidis (2012) or more recently in Kozlova et al. (2016). How- ever, the process of identifying the best perform- ing set of features, is a task that is both expensive and requires a considerable amount of engineering effort (Hardmeier, 2011). On the other hand, ker- nel methods do not require the explicit definition of the features, and rely on the scalar product be- tween vectors for capturing the similarity shared by the sentence pairs. In this paper we present our submission to the WMT17 Shared Task on sentence level Quality Estimation, that makes use …

Content truncated to 500 words