A discriminative framework of integrating translation memory features into SMT

A Discriminative Framework of Integrating Translation Memory Features into SMT Liangyou Li liangyouli@computing.dcu.ie Andy Way away@computing.dcu.ie Qun Liu qliu@computing.dcu.ie CNGL Centre for Global Intelligent Content, School of Computing, Dublin City University, Ireland Abstract Combining Translation Memory (TM) with Statistical Machine Translation (SMT) together has been demonstrated to be beneficial. In this paper, we present a discriminative framework which can integrate TM into SMT by incorporating TM-related feature functions. Experi- ments on English–Chinese and English–French tasks show that our system using TM feature functions only from the best fuzzy match performs significantly better than the baseline phrase- based system on both tasks, and our discriminative model achieves comparable results to those of an effective generative model which uses similar features. Furthermore, with the capacity of handling a large amount of features in the discriminative framework, we propose a method to efficiently use multiple fuzzy matches which brings more feature functions and further signifi- cantly improves our system. 1 Introduction Translation Memory (TM) has been widely used to assist human translators. It provides the most similar source sentence in the database together with the target translation as the reference to a human for post-editing. As TM stores legacy translations, it can give high quality and consistent translations for repetitive materials. However, it performs badly when there are no highly similar matches in TM. In contrast, Statistical Machine Translation (SMT) automatically learns several models, such as the translation model (from parallel data) and language model (from the target side of the parallel corpus as well as other monolingual data), and uses them to translate a new sentence. The translation is produced by maximizing a weighted combination of these models. Given a large amount of data, SMT can generate better results for unseen sentences than TM. However, unless sentence-caching is utilised, it treats a seen sentence (such as a sentence in the training data) as unseen. Clearly, TM and SMT complement one another on matched and unmatched segments, so both are receiving increasing attention from translators and researchers, who would like to combine TM and SMT together to obtain better translation quality with methods such as system recommendation (He et al., 2010a,b) or using fragments from TM in SMT (Bic¸ici and Dymetman, 2008; Koehn and Senellart, 2010; Ma et al., 2011; Wang et al., 2013) This paper is focused on integrating TM into SMT to improve translation quality. We present a discriminative framework which directly integrates TM-related feature functions into SMT. In this paper, we change features extracted from TM which are defined in a generative Al-Onaizan & Simard (Eds.) Proceedings of AMTA 2014, vol. 1: MT Researchers Vancouver, BC © The Authors 249 model (Wang et al., 2013) to feature functions and add them into the phrase-based transla- tion model. Experiments on English–Chinese and English–French tasks show that our method achieves comparable results with Wang et al. (2013), and is significantly better than the baseline phrase-based system. In addition, we present a method to incorporate multiple fuzzy matches into our system, which brings …

Content truncated to 500 words