Training MRF-Based Phrase Translation Models using Gradient Ascent

Proceedings of NAACL-HLT 2013, pages 450–459, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Training MRF-Based Phrase Translation Models using Gradient Ascent Jianfeng Gao Microsoft Research Redmond, WA, USA jfgao@microsoft.com Xiaodong He Microsoft Research Redmond, WA, USA xiaohe@microsoft.com Abstract This paper presents a general, statistical framework for modeling phrase translation via Markov random fields. The model al- lows for arbituary features extracted from a phrase pair to be incorporated as evidence. The parameters of the model are estimated using a large-scale discriminative training approach that is based on stochastic gradi- ent ascent and an N-best list based expected BLEU as the objective function. The model is easy to be incoporated into a standard phrase-based statistical machine translation system, requiring no code change in the runtime engine. Evaluation is performed on two Europarl translation tasks, German- English and French-English. Results show that incoporating the Markov random field model significantly improves the perfor- mance of a state-of-the-art phrase-based machine translation system, leading to a gain of 0.8-1.3 BLEU points. 1 Introduction The phrase translation model, also known as the phrase table, is one of the core components of a phrase-based statistical machine translation (SMT) system. The most common method of constructing the phrase table takes a two-phase approach. First, the bilingual phrase pairs are extracted heuristical- ly from an automatically word-aligned training da- ta. The second phase is parameter estimation, where each phrase pair is assigned with some scores that are estimated based on counting of words or phrases on the same word-aligned train- ing data. There has been a lot of research on improving the quality of the phrase table using more princi- pled methods for phrase extraction (e.g., Lamber and Banchs 2005), parameter estimation (e.g., Wuebker et al. 2010; He and Deng 2012), or both (e.g., Marcu and Wong 2002; Denero et al. 2006). The focus of this paper is on the parameter estima- tion phase. We revisit the problem of scoring a phrase translation pair by developing a new phrase translation model based on Markov random fields (MRFs) and large-scale discriminative training. We strive to address the following three primary concerns. First of all, instead of parameterizing a phrase translation pair using a set of scoring functions that are learned independently (e.g., phrase translation probabilities and lexical weights) we use a general, statistical framework in which arbitrary features extracted from a phrase pair can be incorporated to model the translation in a unified way. To this end, we propose the use of a MRF model. Second, because the phrase model has to work with other component models in an SMT system in order to produce good translations and the quality of translation is measured via BLEU score, it is de- sirable to optimize the parameters of the phrase model jointly with other component models with respect to an objective function that is closely re- lated to the evaluation metric under consideration, i.e., BLEU in this paper. To this end, we resort to a large-scale discriminative training approach, …

Content truncated to 500 words