PRHLT Submission to the WMT12 Quality Estimation Task

Proceedings of the 7th Workshop on Statistical Machine Translation, pages 104–108, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics PRHLT Submission to the WMT12 Quality Estimation Task Jes´us Gonz´alez Rubio and Alberto Sanchis and Francisco Casacuberta D. Sistemas Inform´aticos y Computaci´on Universitat Polit`ecnica de Val`encia Camino de vera s/n, 46022, Valencia, Spain {jegonzalez,josanna,fcn}@dsic.upv.es Abstract This is a description of the submissions made by the pattern recognition and human lan- guage technology group (PRHLT) of the Uni- versitat Polit`ecnica de Val`encia to the qual- ity estimation task of the seventh workshop on statistical machine translation (WMT12). We focus on two different issues: how to ef- fectively combine subsequence-level features into sentence-level features, and how to select the most adequate subset of features. Results showed that an adequate selection of a subset of highly discriminative features can improve efficiency and performance of the quality esti- mation system. 1 Introduction Quality estimation (QE) (Ueffing et al., 2003; Blatz et al., 2004; Sanchis et al., 2007; Specia and Farzin- dar, 2010) is a topic of increasing interest in machine translation (MT). It aims at providing a quality indi- cator for unseen translations at various granularity levels. Different from MT evaluation, QE do not rely on reference translations and is generally ad- dressed using machine learning techniques to pre- dict quality scores. Our main focus in this article is in the combi- nation of subsequence features into sentence fea- tures, and in the selection of a subset of relevant fea- tures to improve performance and efficiency. Sec- tion 2 describes the features and the learning algo- rithm used in the experiments. Section 3 describe two different approaches implemented to select the best-performing subset of features. Section 4 dis- plays the results of the experimentation intended to determine the optimal setup to train our final sub- mission. Finally, section 5 summarizes the submis- sion and discusses the results. 2 Features and Learning Algorithm 2.1 Available Sources of Information The WMT12 QE task is carried out on English– Spanish news texts produced by a phrase-based MT system. As training data we are given 1832 trans- lations manually annotated for quality in terms of post-editing effort (scores in the range [1, 5]), to- gether with their source sentences, decoding in- formation, reference translations, and post-edited translations. Additional training data can be used, as deemed appropriate. Any of these information sources can be used to extract the features, however, test data consists only on source sentence, transla- tion, and search information. Thus, features were extracted from the sources of information available in test data only. Additionally, we compute some extra features from the WMT12 translation task (WMT12TT) training data. 2.2 Features We extracted a total of 475 features classified into sentence-level and subsequence-level features. We considered subsequences of sizes one to four. Sentence-level features • Source and target sentence lengths, and ratio. • Proportion of dead nodes in the search graph. • Number of source phrases. • Number and average size of the translation op- tions under …

Content truncated to 500 words