Can Translation Memories afford not to use paraphrasing ?

Can Translation Memories afford not to use paraphrasing? Rohit Gupta1, Constantin Or˘asan1, Marcos Zampieri2,3, Mihaela Vela2, Josef van Genabith2,3 1Research Group in Computational Linguistics, University of Wolverhampton, UK 2Saarland University, Germany 3German Research Center for Artificial Intelligence (DFKI) {r.gupta, c.orasan}@wlv.ac.uk {marcos.zampieri, m.vela}@uni-saarland.de josef.van genabith@dfki.de Abstract This paper investigates to what extent the use of paraphrasing in translation mem- ory (TM) matching and retrieval is use- ful for human translators. Current trans- lation memories lack semantic knowledge like paraphrasing in matching and re- trieval. Due to this, paraphrased seg- ments are often not retrieved. Lack of se- mantic knowledge also results in inappro- priate ranking of the retrieved segments. Gupta and Or˘asan (2014) proposed an im- proved matching algorithm which incorpo- rates paraphrasing. Its automatic evalua- tion suggested that it could be beneficial to translators. In this paper we perform an extensive human evaluation of the use of paraphrasing in the TM matching and retrieval process. We measure post-editing time, keystrokes, two subjective evalua- tions, and HTER and HMETEOR to assess the impact on human performance. Our re- sults show that paraphrasing improves TM matching and retrieval, resulting in trans- lation performance increases when trans- lators use paraphrase enhanced TMs. 1 Introduction One of the core features of a TM system is the retrieval of previously translated similar segments for post-editing in order to avoid translation from scratch when an exact match is not available. How- ever, this retrieval process is still limited to edit- distance based measures operating on surface form c⃝ 2015 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. (or sometimes stem) matching. Most of the com- mercial systems use edit distance (Levenshtein, 1966) or some variation of it, e.g. the open-source TM OmegaT1 uses word-based edit distance with some extra preprocessing. Although these mea- sures provide a strong baseline, they are not suf- ficient to capture semantic similarity between the segments as judged by humans. Gupta and Or˘asan (2014) proposed an edit dis- tance measure which incorporates paraphrasing in the process. In the present paper, we perform a human-centred evaluation to investigate the use of paraphrasing in translation memory matching and retrieval. We use the same system as Gupta and Or˘asan (2014) and investigate the following questions: (1) how much of an improvement can paraphrasing provide in terms of retrieval? (2) What is the quality of the retrieved segments and its impact on the work of human translators? These questions are answered using human centred eval- uations. To the best of our knowledge, this paper presents the first work on assessing the quality of any type of semantically informed TM fuzzy matches based on post-editing time or keystrokes. 2 Related Work Several researchers have used semantic or syntac- tic information in TMs, but their evaluations were shallow and most of the time limited to subjective evaluation carried out by the authors. This makes it hard to judge how much a semantically informed TM matching system can benefit …

Content truncated to 500 words