Unsupervised Discriminative Induction of Synchronous Grammar for Machine Translation

Proceedings of COLING 2012: Technical Papers, pages 2883–2898, COLING 2012, Mumbai, December 2012. Unsupervised Discriminative Induction of Synchronous Grammar for Machine Translation Xinyan Xiao†‡ Deyi Xiong∗ Yang Liu⋆ Qun Liu† Shouxun Lin† †Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China ‡Youdao Inc., Beijing, China ∗Human Language Technology, Institute for Infocomm Research, Singapore ⋆Department of Computer Sci. and Tech., Tsinghua University, Beijing, China xiaoxy@rd.netease.com, dyxiong@i2r.a-star.edu.sg, liuyang2011@tsinghua.edu.cn, liuqun@ict.ac.cn, sxlin@ict.ac.cn ABSTRACT We present a global log-linear model for synchronous grammar induction, which is capable of incorporating arbitrary features. The parameters in the model are trained in an unsuper- vised fashion from parallel sentences without word alignments. To make parameter training tractable, we also propose a novel and efficient cube pruning based synchronous parsing algo- rithm. Using learned synchronous grammar rules with millions of features that contain rule level, word level and translation boundary information, we significantly outperform a compet- itive hierarchical phrased-based baseline system by +1.4 BLEU on average on three NIST test sets. KEYWORDS: synchronous grammar induction, discriminative model, unsupervised learning, machine translation. 2883 1 Introduction In the last decade, statistical machine translation (SMT) has been advanced by expanding the basic unit of translation from word to phrase (Koehn et al., 2003) and grammar (Galley et al., 2006; Liu et al., 2006; Chiang, 2007). Most systems induce synchronous grammars (includ- ing phrases) from parallel corpora using a heuristic two-step pipeline. This pipeline first aligns a parallel corpus at the word level with heuristic word alignment combination strate- gies (e.g., grow-diag-final-and) (Koehn et al., 2003), and then extracts translation rules from word-aligned sentence pairs. It is working well in practice and therefore widely adopted. How- ever, such a pipeline artificially brings an undesirable disconnection between translation model and word alignment model (Blunsom et al., 2009; DeNero and Klein, 2010). Recently, researchers have resorted to principled probabilistic formulations. Various genera- tive models are proposed to learn translation rules directly from sentence pairs without word alignments (Marcu and Wong, 2002; Cherry and Lin, 2007; Zhang et al., 2008; DeNero et al., 2008; Blunsom et al., 2009; Cohn and Blunsom, 2009; Neubig et al., 2011; Levenberg et al., 2012). Due to the independency assumptions in such generative models, it is hard to extend them to incorporate arbitrary features, especially word alignment information as used in the traditional two-step pipeline. As a result, despite theoretical advantages of these models over the two-step pipeline, in practice they can only produce comparable translation quality with the two-step pipeline in some scenarios, and usually even worse results. Yet another alternative for synchronous grammar induction is unsupervised discriminative model. Unsupervised discriminative model can directly learn synchronous grammars in a theoretically justified manner just like generative model. However, the advantage over gen- erative model is that it is able to easily incorporate word alignment information which has been proved useful in the two-step pipeline. In this paper, we propose a global log-linear model (Sec. 2) for the induction of synchronous context free grammar (SCFG) (Chiang, 2007). The log-linear model is …

Content truncated to 500 words