Referential Translation Machines for Predicting Translation Quality

Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 313–321, Baltimore, Maryland USA, June 26–27, 2014. c⃝2014 Association for Computational Linguistics Referential Translation Machines for Predicting Translation Quality Ergun Bic¸ici Centre for Next Generation Localisation School of Computing Dublin City University, Dublin, Ireland. ergun.bicici@computing.dcu.ie Andy Way Centre for Next Generation Localisation School of Computing Dublin City University, Dublin, Ireland. away@computing.dcu.ie Abstract We use referential translation machines (RTM) for quality estimation of translation outputs. RTMs are a computational model for identifying the translation acts between any two data sets with respect to interpre- tants selected in the same domain, which are effective when making monolingual and bilingual similarity judgments. RTMs achieve top performance in automatic, ac- curate, and language independent predic- tion of sentence-level and word-level sta- tistical machine translation (SMT) qual- ity. RTMs remove the need to access any SMT system specific information or prior knowledge of the training data or models used when generating the translations and achieve the top performance in WMT13 quality estimation task (QET13). We im- prove our RTM models with the Parallel FDA5 instance selection model, with ad- ditional features for predicting the trans- lation performance, and with improved learning models. We develop RTM mod- els for each WMT14 QET (QET14) sub- task, obtain improvements over QET13 re- sults, and rank 1st in all of the tasks and subtasks of QET14. 1 Introduction We use referential translation machines (RTM) for quality estimation of translation outputs, which is a computational model for identifying the acts of translation for translating between any given two data sets with respect to a reference corpus se- lected in the same domain. RTMs reduce our de- pendence on any task dependent resource. Predic- tion of translation quality is important because the expected translation performance can help in esti- mating the effort required for correcting the trans- lations during post-editing by human translators. Bicici et al. (2013) develop the Machine Trans- lation Performance Predictor (MTPP), a state-of- the-art, language independent, and SMT system extrinsic machine translation performance predic- tor, which can predict translation quality by look- ing at the test source sentences and becomes the 2nd overall after also looking at the translation outputs as well in QET12 (Callison-Burch et al., 2012). RTMs achieve the top performance in QET13 (Bojar et al., 2013), ranking 1st or 2nd in all of the subtasks. RTMs rank 1st in all of the tasks and subtasks of QET14 (Bojar et al., 2014). Referential translation models (Section 2) present an accurate and language independent so- lution for predicting the performance of natural language tasks such as the quality estimation of translation. We improve our RTM models (Bic¸ici, 2013) by: • using a parameterized, fast implementation of FDA, FDA5, and our Parallel FDA5 in- stance selection model (Bic¸ici et al., 2014), • better modeling of the language in which similarity judgments are made with improved optimization and selection of the LM data, • increased feature set for also modeling the structural properties of sentences, • extended learning …

Content truncated to 500 words