LIG System for WMT13 QE Task: Investigating the Usefulness of Features in Word Confidence Estimation for MT

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 386–391, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics LIG System for WMT13 QE Task: Investigating the Usefulness of Features in Word Confidence Estimation for MT Ngoc-Quang Luong Benjamin Lecouteux LIG, Campus de Grenoble 41, Rue des Math´ematiques, UJF - BP53, F-38041 Grenoble Cedex 9, France {ngoc-quang.luong,laurent.besacier,benjamin.lecouteux}@imag.fr Laurent Besacier Abstract This paper presents the LIG’s systems submitted for Task 2 of WMT13 Qual- ity Estimation campaign. This is a word confidence estimation (WCE) task where each participant was asked to la- bel each word in a translated text as a binary ( Keep/Change) or multi-class (Keep/Substitute/Delete) category. We in- tegrate a number of features of various types (system-based, lexical, syntactic and semantic) into the conventional feature set, for our baseline classifier training. After the experiments with all features, we deploy a “Feature Selection” strategy to keep only the best performing ones. Then, a method that combines multiple “weak” classifiers to build a strong “com- posite” classifier by taking advantage of their complementarity is presented and ex- perimented. We then select the best sys- tems for submission and present the offi- cial results obtained. 1 Introduction Recently Statistical Machine Translation (SMT) systems have shown impressive gains with many fruitful results. While the outputs are more accept- able, the end users still face the need to post edit (or not) an automatic translation. Then, the issue is to be able to accurately identify the correct parts as well as detecting translation errors. If we fo- cus on errors at the word level, the issue is called Word-level Confidence Estimation (WCE). In WMT 2013, a shared task about quality esti- mation is proposed. This quality estimation task is proposed at two levels: word-level and sentence- level. Our work focuses on the word-level qual- ity estimation (named Task 2). The objective is to highlight words needing post-edition and to detect parts of the sentence that are not reliable. For the task 2, participants produce for each token a label according to two sub-tasks: • a binary classification: good (keep) or bad (change) label • a multi-class classification: the label refers to the edit action needed for the token (i.e. keep, delete or substitute). Various approaches have been proposed for WCE: Blatz et al. (2003) combine several features using neural network and naive Bayes learning al- gorithms. One of the most effective feature combi- nations is the Word Posterior Probability (WPP) as proposed by Ueffing et al. (2003) associated with IBM-model based features (Blatz et al., 2004). Ueffing and Ney (2005) propose an approach for phrase-based translation models: a phrase is a se- quence of contiguous words and is extracted from word-aligned bilingual training corpus. The con- fidence value of each word is then computed by summing over all phrase pairs in which the tar- get part contains this word. Xiong et al. (2010) integrate target word’s Part-Of-Speech (POS) and train them by Maximum Entropy Model, allow- ing significative gains compared to WPP …

Content truncated to 500 words