LORIA System for the WMT12 Quality Estimation Shared Task

Proceedings of the 7th Workshop on Statistical Machine Translation, pages 114–119, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics LORIA System for the WMT12 Quality Estimation Shared Task Langlois David langlois@loria.fr Raybaud Sylvain LORIA, Universit´e de Lorraine 615 rue du Jardin Botanique 54602 Villers les Nancy, France raybauds@loria.fr Sma¨ıli Kamel smaili@loria.fr Abstract In this paper we present the system we sub- mitted to the WMT12 shared task on Quality Estimation. Each translated sentence is given a score between 1 and 5. The score is ob- tained using several numerical or boolean fea- tures calculated according to the source and target sentences. We perform a linear regres- sion of the feature space against scores in the range [1:5]. To this end, we use a Support Vec- tor Machine. We experiment with two kernels: linear and radial basis function. In our submis- sion we use the features from the shared task baseline system and our own features. This leads to 66 features. To deal with this large number of features, we propose an in-house feature selection algorithm. Our results show that a lot of information is already present in baseline features, and that our feature selec- tion algorithm discards features which are lin- early correlated. 1 Introduction Machine translation systems are not reliable enough to be used directly. They can only be used to grasp the general meaning of texts or help human transla- tors. Confidence measures detect erroneous words or sentences. Such information could be useful for users to decide whether or not to post-edit translated sentences (Specia, 2011; Specia et al., 2010) or se- lect documents mostly correctly translated (Soricut and Echihabi, 2010). Moreover, it is possible to use confidence measures to compare outputs from dif- ferent systems and to recommend the best one (He et al., 2010). One can also imagine that confidence measures at word-level could be also useful for a machine translation system to automatically correct parts of output: for example, a translation system translates the source sentence, then, this output is translated with another translation system (Simard et al., 2007). This last step could be driven by confi- dence measures. In previous works (Raybaud et al., 2011; Raybaud et al., 2009a; Raybaud et al., 2009b) we used state- of-the-art features to predict the quality of a transla- tion at sentence- and word-level. Moreover, we pro- posed our own features based on previous works on cross-lingual triggers (Lavecchia et al., 2008; Latiri et al., 2011). We evaluated our work in terms of Dis- crimination Error Trade-off, Equal Error Rate and Normalised Mutual Information. In this article, we compare the features used in the shared task baseline system and our own features. This leads to 66 features which will be detailed in sections 3 and 4. We therefore deal with many fea- tures. We used a machine learning approach to per- form regression of the feature space against scores given by humans. Machine learning algorithms may not efficiently deal with high dimensional spaces. Moreover, some features …

Content truncated to 500 words