Integrating a Large, Monolingual Corpus as Translation Memory into Statistical Machine translation

Integrating a Large, Monolingual Corpus as Translation Memory into Statistical Machine Translation Katharina W¨aschle and Stefan Riezler Department of Computational Linguistics Heidelberg University 69120 Heidelberg, Germany {waeschle, riezler}@cl.uni-heidelberg.de Abstract Translation memories (TM) are widely used in the localization industry to im- prove consistency and speed of human translation. Several approaches have been presented to integrate the bilingual trans- lation units of TMs into statistical ma- chine translation (SMT). We present an extension of these approaches to the in- tegration of partial matches found in a large, monolingual corpus in the target language, using cross-language informa- tion retrieval (CLIR) techniques. We use locality-sensitive hashing (LSH) for ef- ficient coarse-grained retrieval of match candidates, which are then filtered by fine- grained fuzzy matching, and finally used to re-rank the n-best SMT output. We show consistent and significant improvements over a state-of-the-art SMT system, across different domains and language pairs on tens of millions of sentences. 1 Introduction A translation memory (TM) is a computational tool used by professional translators to speed up translation of repetitive texts. At its core is a database, in which source and target of previ- ously translated segments of text are stored. TMs are capable of retrieving not only exact, but also partial matches, where only a certain percentage of source words overlap with the query, called fuzzy matches. A computer-assisted translation (CAT) tool presents possible matches found in the c⃝ 2015 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. database to a user, if the match is considered sim- ilar enough to the current source sentence. Even if the presented target sentence is not a perfect translation, a fuzzy match can be a good starting point for the translation of the current sentence and reduce translation time and effort. Furthermore, the approach can help with translation consistency and terminology control. In contrast to statistical machine translation (SMT), TM tools are widely used in the translation industry, since the results presented to the translator are fluent translations. They are especially successful for translation of texts from repetitive domains, e.g. technical docu- ments such as IT manuals, that are the predominant use case in the localization industry. The idea of combining the strengths of TM and SMT tools has been successfully explored in re- cent years. In this paper, we extend these ap- proaches to the integration of a large, monolingual corpus in the target language as a TM into an SMT system using cross-language information retrieval (CLIR). Our approach utilizes locality-sensitive hashing (LSH) as an efficient coarse retrieval tech- nique to select candidate translations. In a next step, search is performed at a finer-grained level using distance metrics customary in CAT. Given a match, our model re-ranks the n-best list out- put by an SMT decoder using features modeling the closeness of the hypothesis and the target of the TM match. Since our approach does not rely on an alignment between source and target side of the TM …

Content truncated to 500 words