UAlacant word-level machine translation quality estimation system at WMT 2015

Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 309–315, Lisboa, Portugal, 17-18 September 2015. c⃝2015 Association for Computational Linguistics. UAlacant word-level machine translation quality estimation system at WMT 2015 Miquel Espl`a-Gomis Felipe S´anchez-Mart´ınez Mikel L. Forcada Departament de Llenguatges i Sistemes Inform`atics Universitat d’Alacant, E-03071 Alacant, Spain {mespla,fsanchez,mlf}@dlsi.ua.es Abstract This paper describes the Universitat d’Alacant submissions (labelled as UAla- cant) for the machine translation quality estimation (MTQE) shared task in WMT 2015, where we participated in the word- level MTQE sub-task. The method we used to produce our submissions uses external sources of bilingual information as a black box to spot sub-segment correspondences between a source segment S and the trans- lation hypothesis T produced by a machine translation system. This is done by seg- menting both S and T into overlapping sub- segments of variable length and translating them in both translation directions, using the available sources of bilingual informa- tion on the fly. For our submissions, two sources of bilingual information were used: machine translation (Apertium and Google Translate) and the bilingual concordancer Reverso Context. After obtaining the sub- segment correspondences, a collection of features is extracted from them, which are then used by a binary classifer to obtain the final “GOOD” or “BAD” word-level qual- ity labels. We prepared two submissions for this year’s edition of WMT 2015: one using the features produced by our system, and one combining them with the baseline features published by the organisers of the task, which were ranked third and first for the sub-task, respectively. 1 Introduction Machine translation (MT) post-editing is nowadays an indispensable step that allows to use machine translation for dissemination. Consequently, MT quality estimation (MTQE) (Blatz et al., 2004; Spe- cia et al., 2010; Specia and Soricut, 2013) has emerged as a mean to minimise the post-editing ef- fort by developing techniques that allow to estimate the quality of the translation hypotheses produced by an MT system. In order to boost the scientific efforts on this problem, the WMT 2015 MTQE shared task proposes three tasks that allow to com- pare different approaches at three different levels: segment-level (sub-task 1), word-level (sub-task 2), and document-level (sub-task 3). Our submissions tackle the word-level MTQE sub-task, which proposes a framework for evalu- ating and comparing different approaches. This year, the sub-task used a dataset obtained by trans- lating segments in English into Spanish using MT. The task consists in identifying which words in the translation hypothesis had to be post-edited and which of them had to be kept unedited by applying the labels “BAD” and “GOOD”, respectively. In this paper we describe the approach behind the two submissions of the Universitat d’Alacant team to this sub-task. For our submissions we applied the approach proposed by Espl`a-Gomis et al. (2015b), who use black-box bilingual resources from the Internet for word-level MTQE. In particular, we combined two on-line MT systems, Apertium1 and Google Translate,2 and the bilingual concordancer Reverso Context3 to spot sub-segment correspon- dences between a sentence S …

Content truncated to 500 words