LORIA System for the WMT13 Quality Estimation Shared Task

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 380–385, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics LORIA System for the WMT13 Quality Estimation Shared Task Langlois David LORIA (Universit´e de Lorraine, INRIA, CNRS) 615 rue du Jardin Botanique, 54602 Villers les Nancy, France david.langlois@loria.fr Sma¨ıli Kamel LORIA (Universit´e de Lorraine, INRIA, CNRS) 615 rue du Jardin Botanique, 54602 Villers les Nancy, France kamel.smaili@loria.fr Abstract In this paper we present the system we submitted to the WMT13 shared task on Quality Estimation. We participated in the Task 1.1. Each translated sentence is given a score between 0 and 1. The score is obtained by using several numeri- cal or boolean features calculated accord- ing to the source and target sentences. We perform a linear regression of the feature space against scores in the range [0..1]. To this end, we use a Support Vector Machine with 66 features. In this paper, we propose to increase the size of the training corpus. For that, we use the post-edited and refer- ence corpora during the training step. We assign a score to each sentence of these corpora. Then, we tune these scores on a development corpus. This leads to an im- provement of 10.5% on the development corpus, in terms of Mean Average Error, but achieves only a slight improvement on the test corpus. 1 Introduction In the scope of Machine Translation (MT), Qual- ity Estimation (QE) is the task consisting to evalu- ate the translation quality of a sentence or a docu- ment. This process may be useful for post-editors to decide or not to revise a sentence produced by a MT system (Specia, 2011; Specia et al., 2010). Moreover, it can be useful to decide if a translated document can be broadcasted or not (Soricut and Echihabi, 2010). The most obvious way to give a score to a translated sentence consists in using a machine learning approach. This approach is su- pervised: experts are asked to score translated sen- tences and with the obtained material, one learns a prediction model of scores. The main drawback of the machine learning approach is that it is super- vised and requires huge data. To score a sentence is time-consuming. Moreau et al. in (Moreau and Vogel, 2012) dealt with this issue by proposing un- supervised similarity measures. In fact, the score of a translated sentence is defined by a measure giving the distance between it and the contents of an external corpus. The authors improve the re- sults of the supervised approach but this method can be used only in the ranking task. Raybaud et al. (Raybaud et al., 2011) proposed a method to add errors in reference sentences (deletion, sub- stitution, insertion). By this way, they build addi- tional corpus in which each word can be associated with a label correct/not correct. But, it is not pos- sible to predict the translation quality of sentences including these erroneous words. In this paper, we propose to increase the size of …

Content truncated to 500 words