The repetition rate of text as a predictor of the effectiveness of machine translation adaptation

The Repetition Rate of Text as a Predictor of the Effectiveness of Machine Translation Adaptation Mauro Cettolo cettolo@fbk.eu Nicola Bertoldi bertoldi@fbk.eu Marcello Federico federico@fbk.eu FBK, Fondazione Bruno Kessler, 38123 Povo, Trento, Italy Abstract Since the effectiveness of MT adaptation relies on the text repetitiveness, the question on how to measure repetitions in a text naturally arises. This work deals with the issue of looking for and evaluating text features that might help the prediction of the impact of MT adaptation on translation quality. In particular, the repetition rate metric, we recently proposed, is compared to other features employed in very related NLP tasks. The comparison is carried out through a regression analysis between feature values and MT performance gains by dynamically adapted versus non-adapted MT engines, on five different translation tasks. The main outcome of ex- periments is that the repetition rate correlates better than any other considered feature with the MT gains yielded by the online adaptation, although using all features jointly results in better predictions than with any single feature. 1 Introduction Language and content repetitiveness1 is a key factor for the successful deployment of translation memories (TMs) (Somers, 2003) as well as statistical machine translation (MT) (Koehn, 2010). The capability of a TM to provide useful translation suggestions for a text segment relies on the chance of finding segments with very similar content – i.e. with a significant percentage of overlapping words – inside a large repository of already translated texts. On the other hand, statistical MT also relies on the assumption that the segment to be translated shares with the training data a significant amount of patterns, from single words to groups of words. Advances on the integration of human post-editing into MT have recently revealed the po- tential of incremental and online MT adaptation. While doing their job, post-editors are also generating fresh in-domain training data. Adding these data to the MT engine would for in- stance mean letting MT focus on the lexical choices of the post-editor or even avoid future translation mistakes. Given that the internal knowledge of statistical MT is mainly represented by translation patterns, the potential impact of MT adaptation strictly depends on how much such type of patterns will re-occur in future sentences. We can hence claim that text repet- itiveness is a precondition for the effectiveness of MT adaptation in general, whether this is performed on sentence batches as for incremental adaptation, or single sentence as in online adaptation. Thus, the question becomes how to measure repetitions in a text in a way to help the prediction2 of the impact of MT adaptation on translation quality for that text. Such a prediction 1In this paper the word repetitiveness is not used with a negative meaning, e.g. boring, unpleasant. 2Although sometimes they are given slightly different meaning, in this work we consider prediction and forecast as synonyms. Al-Onaizan & Simard (Eds.) Proceedings of AMTA 2014, vol. 1: MT Researchers Vancouver, BC © The Authors 166 could of course be useful …

Content truncated to 500 words