Dynamically Integrating Cross-Domain Translation Memory into Phrase-Based Machine Translation during Decoding

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 398–408, Dublin, Ireland, August 23-29 2014. Dynamically Integrating Cross-Domain Translation Memory into Phrase-Based Machine Translation during Decoding Kun Wang† Chengqing Zong† Keh-Yih Su‡ †National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China ‡Institute of Information Science, Academia Sinica, Taiwan †{kunwang, cqzong}@nlpr.ia.ac.cn ‡kysu@iis.sinica.edu.tw Abstract Our previous work focuses on combining translation memory (TM) and statistical machine translation (SMT) when the TM database and the SMT training set are the same. However, the TM database will deviate from the SMT training set in the real task when time goes by. In this work, we concentrate on the task when the TM database and the SMT training set are different and even from different domains. Firstly, we dynamically merge the matched TM phrase-pairs into the SMT phrase table to meet the real application. Secondly, we propose an improved integrated model to distinguish the original and the new- ly-added phrase-pairs. Thirdly, a simple but effective TM adaptation method is adopted to favor the consistent translations in cross-domain test. Our experiments have shown that merging the TM phrase- pairs achieves significant improvements. Furthermore, the proposed approaches are significantly better than the TM, the SMT and previous integration works for both in-domain and cross-domain tests. 1 Introduction Since the translation memory (TM) system and the statistical machine translation (SMT) system com- plement each other in those matched sub-segments and unmatched sub-segments (Wang et al., 2013), combining them can improve the output quality significantly, especially when high-similarity fuzzy matches are available. Therefore, combining TM and SMT is drawing more and more attention in re- cent years (He et al., 2010a; 2010b; 2011; Koehn and Senellart, 2010; Zhechev and van Genabith, 2010; Ma et al., 2011; Dara et al., 2013; Wang et al., 2013). Those previous works on combining TM and SMT can be classified into four categories: (1) select- ing the better translation sentence from TM and SMT (He et al., 2010a; 2010b; Dara et al., 2013); (2) incorporating TM matched sub-segments into SMT in a pipelined manner (Koehn and Senellart, 2010; He et al., 2011; Ma et al., 2011); (3) only enhancing the SMT phrase table with new TM phrase-pairs (Biçici and Dymetman, 2008; Simard and Isabelle, 2009); and (4) incorporating the associated TM information with each source phrase to guide the SMT decoding (Wang et al., 2013). However, all previous works mentioned above only focus on the case in which the TM database and the SMT training set share the same data-set. Nonetheless, in real applications, the TM database will deviate from the SMT training set when time goes by, because the TM database will be dynamically enlarged when more translations are generated by the human translator. Therefore, this paper will con- centrate on a more realistic case, in which the TM database and the SMT training set are different and even from different domains. When the TM database and the SMT training set share the …

Content truncated to 500 words