Max-Margin Synchronous Grammar Induction for Machine Translation

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 255–264, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Max-Margin Synchronous Grammar Induction for Machine Translation Xinyan Xiao and Deyi Xiong∗ School of Computer Science and Technology Soochow University Suzhou 215006, China xyxiao.cn@gmail.com, dyxiong@suda.edu.cn Abstract Traditional synchronous grammar induction estimates parameters by maximizing likeli- hood, which only has a loose relation to trans- lation quality. Alternatively, we propose a max-margin estimation approach to discrim- inatively inducing synchronous grammars for machine translation, which directly optimizes translation quality measured by BLEU. In the max-margin estimation of parameters, we only need to calculate Viterbi translations. This further facilitates the incorporation of various non-local features that are defined on the target side. We test the effectiveness of our max-margin estimation framework on a com- petitive hierarchical phrase-based system. Ex- periments show that our max-margin method significantly outperforms the traditional two- step pipeline for synchronous rule extraction by 1.3 BLEU points and is also better than pre- vious max-likelihood estimation method. 1 Introduction Synchronous grammar induction, which refers to the process of learning translation rules from bilin- gual corpus, still remains an open problem in sta- tistical machine translation (SMT). Although state- of-the-art SMT systems model the translation pro- cess based on synchronous grammars (including bilingual phrases), most of them still learn trans- lation rules via a pipeline with word-based heuris- tics (Koehn et al., 2003). This pipeline first builds word alignments using heuristic combination strate- gies, then heuristically extracts rules that are consis- tent with word alignments. Such heuristic pipeline ∗Corresponding author is not elegant theoretically. It brings an undesirable gap that separates modeling and learning in an SMT system. Therefore, researchers have proposed alternative approaches to learning synchronous grammars di- rectly from sentence pairs without word alignments, via generative models (Marcu and Wong, 2002; Cherry and Lin, 2007; Zhang et al., 2008; DeNero et al., 2008; Blunsom et al., 2009; Cohn and Blun- som, 2009; Neubig et al., 2011; Levenberg et al., 2012) or discriminative models (Xiao et al., 2012). Theoretically, these approaches describe how sen- tence pairs are generated by applying sequences of synchronous rules in an elegant way. However, they learn synchronous grammars by maximizing likeli- hood,1 which only has a loose relation to transla- tion quality (He and Deng, 2012). Moreover, gen- erative models are normally hard to be extended to incorporate useful features, and the discriminative synchronous grammar induction model proposed by Xiao et al. (2012) only incorporates local features defined on parse trees of the source language. Non- local features, which encode information from parse trees of the target language, have never been ex- ploited before due to the computational complexity of normalization in max-likelihood estimation. Consequently, we would like to learn syn- chronous grammars in a discriminative way that can directly maximize the end-to-end translation quality measured by BLEU (Papineni et al., 2002), and is also able to incorporate non-local features from tar- get parse trees. We thus propose a max-margin …

Content truncated to 500 words