Relevance Ranking for Translated Texts

Relevance Ranking for Translated Texts Marco Turchi, Josef Steinberger European Commission JRC IPSC - GlobeSec Via Fermi 2749, 21020 Ispra (VA), Italy name.surname@jrc.ec.europa.eu Lucia Specia Department of Computer Science University of Sheffield Regent Court, 211 Portobello Sheffield, S1 4DP, UK L.Specia@dcs.shef.ac.uk Abstract The usefulness of a translated text for gist- ing purposes strongly depends on the over- all translation quality of the text, but espe- cially on the translation quality of the most informative portions of the text. In this paper we address the problems of rank- ing translated sentences within a document and ranking translated documents within a set of documents on the same topic accord- ing to their informativeness and translation quality. An approach combining quality estimation and sentence ranking methods is used. Experiments with French-English translation using four sets of news com- mentary documents show promising re- sults for both sentence and document rank- ing. We believe that this approach can be useful in several practical scenarios where translation is aimed at gisting, such as mul- tilingual media monitoring and news anal- ysis applications. 1 Introduction Reading and understanding the main ideas behind documents written in different languages can be necessary or desirable in a number of scenarios. Existing online translation systems such as Google Translate and Bing Translator1 serve to this pur- pose, mitigating the language barrier effects. De- spite the large improvements in translation qual- ity in recent years, translated documents are still affected by the presence of sentences which are not correctly translated and in the extreme case, c⃝ 2012 European Association for Machine Translation. 1translate.google.com/ and www. microsofttranslator.com/ whose original meaning has been lost. These sen- tences can compromise the readability and reliabil- ity of translated documents, especially if they are the ones that should convey the most important in- formation in the document. Quality estimation methods can flag incorrect translations without access to reference sentences, however the informativeness of these sentences is not taken into account. On the other hand, sentence ranking methods are able to identify the most rel- evant sentences in a given language for tasks such as document summarisation. However, the perfor- mance of sentence ranking algorithms for machine translated texts can be significantly degraded due to the introduction of errors by the translation pro- cess, as it has been shown for other language pro- cessing tasks, e.g. in information retrieval (Savoy and Dolamic, 2009). Moreover, particularly in the case of supervised ranking methods, these may only be available for the source language. In this paper we propose combining quality esti- mation and relevance sentence ranking methods in order to identify the most relevant translated texts. We experiment with two ranking tasks: • The ranking of translated sentences within a document; and • The ranking of documents within a set of doc- uments on the same topic. An evaluation with French-English translations in groups of news commentary documents in dif- ferent domains has shown promising results for both sentence and document ranking. 2 Related work A considerable amount of …

Content truncated to 500 words