Online Multitask Learning for Machine Translation Quality Estimation

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 219–228, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Online Multitask Learning for Machine Translation Quality Estimation Jos´e G. C. de Souza(1,2), Matteo Negri(1), Elisa Ricci(1), Marco Turchi(1) (1) FBK - Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy (2) University of Trento, Italy {desouza,negri,eliricci,turchi}@fbk.eu Abstract We present a method for predicting ma- chine translation output quality geared to the needs of computer-assisted translation. These include the capability to: i) con- tinuously learn and self-adapt to a stream of data coming from multiple translation jobs, ii) react to data diversity by ex- ploiting human feedback, and iii) leverage data similarity by learning and transferring knowledge across domains. To achieve these goals, we combine two supervised machine learning paradigms, online and multitask learning, adapting and unifying them in a single framework. We show the effectiveness of our approach in a re- gression task (HTER prediction), in which online multitask learning outperforms the competitive online single-task and pooling methods used for comparison. This in- dicates the feasibility of integrating in a CAT tool a single QE component capa- ble to simultaneously serve (and continu- ously learn from) multiple translation jobs involving different domains and users. 1 Introduction Even if not perfect, machine translation (MT) is now getting reliable enough to support and speed- up human translation. Thanks to this progress, the work of professional translators is gradually shifting from full translation from scratch to MT post-editing. Advanced computer-assisted trans- lation (CAT) tools1 provide a natural framework for this activity by proposing, for each segment in a source document, one or more suggestions ob- tained either from a translation memory (TM) or from an MT engine. In both cases, accurate mech- anisms to indicate the reliability of a suggestion 1See for instance the open source MateCat tool (Federico et al., 2014). are extremely useful to let the user decide whether to post-edit a given suggestion or ignore it and translate the source segment from scratch. How- ever, while scoring TM matches relies on standard methods based on fuzzy matching, predicting the quality of MT suggestions at run-time and without references is still an open issue. This is the goal of MT quality estimation (QE), which aims to predict the quality of an automatic translation as a function of the estimated number of editing operations or the time required for man- ual correction (Specia et al., 2009; Soricut and Echihabi, 2010; Bach et al., 2011; Mehdad et al., 2012). So far, QE has been mainly approached in controlled settings where homogeneous train- ing and test data is used to learn and evaluate static predictors. Cast in this way, however, it does not fully reflect (nor exploit) the working conditions posed by the CAT framework, in which: 1. The QE module is exposed to a continuous stream of data. The amount of such data and the tight schedule of …

Content truncated to 500 words