Adaptive Quality Estimation for Machine Translation

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 710–720, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Adaptive Quality Estimation for Machine Translation Marco Turchi(1) Antonios Anastasopoulos(3) Jos´e G. C. de Souza(1,2) Matteo Negri(1) (1) FBK - Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy (2) University of Trento, Italy (3) National Technical University of Athens, Greece {turchi,desouza,negri}@fbk.eu anastasopoulos.ant@gmail.com Abstract The automatic estimation of machine translation (MT) output quality is a hard task in which the selection of the appro- priate algorithm and the most predictive features over reasonably sized training sets plays a crucial role. When moving from controlled lab evaluations to real-life sce- narios the task becomes even harder. For current MT quality estimation (QE) sys- tems, additional complexity comes from the difficulty to model user and domain changes. Indeed, the instability of the sys- tems with respect to data coming from dif- ferent distributions calls for adaptive so- lutions that react to new operating con- ditions. To tackle this issue we propose an online framework for adaptive QE that targets reactivity and robustness to user and domain changes. Contrastive exper- iments in different testing conditions in- volving user and domain changes demon- strate the effectiveness of our approach. 1 Introduction After two decades of steady progress, research in statistical machine translation (SMT) started to cross its path with translation industry with tan- gible mutual benefit. On one side, SMT research brings to the industry improved output quality and a number of appealing solutions useful to increase translators’ productivity. On the other side, the market needs suggest concrete problems to solve, providing real-life scenarios to develop and eval- uate new ideas with rapid turnaround. The evolu- tion of computer-assisted translation (CAT) envi- ronments is an evidence of this trend, shown by the increasing interest towards the integration of suggestions obtained from MT engines with those derived from translation memories (TMs). The possibility to speed up the translation pro- cess and reduce its costs by post-editing good- quality MT output raises interesting research chal- lenges. Among others, these include deciding what to present as a suggestion, and how to do it in the most effective way. In recent years, these issues motivated research on automatic QE, which addresses the problem of estimating the quality of a translated sentence given the source and without access to reference translations (Blatz et al., 2003; Specia et al., 2009; Mehdad et al., 2012). Despite the substantial progress done so far in the field and in success- ful evaluation campaigns (Callison-Burch et al., 2012; Bojar et al., 2013), focusing on concrete market needs makes possible to further define the scope of research on QE. For instance, moving from controlled lab testing scenarios to real work- ing environments poses additional constraints in terms of adaptability of the QE models to the vari- able conditions of a translation job. Such variabil- ity is due to two main reasons: 1. The notion of MT output quality is highly …

Content truncated to 500 words