A Bayesian Model of Syntax-Directed Tree to String Grammar Induction

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 352–361, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP A Bayesian Model of Syntax-Directed Tree to String Grammar Induction Trevor Cohn and Phil Blunsom School of Informatics University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB Scotland, United Kingdom {tcohn,pblunsom}@inf.ed.ac.uk Abstract Tree based translation models are a com- pelling means of integrating linguistic in- formation into machine translation. Syn- tax can inform lexical selection and re- ordering choices and thereby improve translation quality. Research to date has focussed primarily on decoding with such models, but less on the difficult problem of inducing the bilingual grammar from data. We propose a generative Bayesian model of tree-to-string translation which induces grammars that are both smaller and pro- duce better translations than the previous heuristic two-stage approach which em- ploys a separate word alignment step. 1 Introduction Many recent advances in statistical machine trans- lation (SMT) are a result of the incorporation of syntactic knowledge into the translation process (Marcu et al., 2006; Zollmann and Venugopal, 2006). This has been facilitated by the use of syn- chronous grammars to model translation as a gen- erative process over pairs of strings in two lan- guages. Such models are particularly attractive for translating between languages with divergent word orders, such as Chinese and English, where syntax-inspired translation rules can succinctly de- scribe the requisite reordering operations. In con- trast, standard phrase-based models (Koehn et al., 2003) assume a mostly monotone mapping be- tween source and target, and therefore cannot adequately model these phenomena. Currently the most successful paradigm for the use of syn- chronous grammars in translation is that of string- to-tree transduction (Galley et al., 2004; Zollmann and Venugopal, 2006; Galley et al., 2006; Marcu et al., 2006). In this case a grammar is extracted from a parallel corpus, with strings on its source side and syntax trees on its target side, which is then used to translate novel sentences by perform- ing inference over the space of target syntax trees licensed by the grammar. To date grammar-based translation models have relied on heuristics to extract a grammar from a word-aligned parallel corpus. These heuristics are extensions of those developed for phrase-based models (Koehn et al., 2003), and involve sym- metrising two directional word alignments fol- lowed by a projection step which uses the align- ments to find a mapping between source words and nodes in the target parse trees (Galley et al., 2004). However, such approaches leave much to be de- sired. Word-alignments rarely factorise cleanly with parse trees (i.e., alignment points cross con- stituent structures), resulting in large and implau- sible translation rules which generalise poorly to unseen data (Fossum et al., 2008). The principal reason for employing a grammar based formal- ism is to induce rules which capture long-range reorderings between source and target. However if the grammar itself is extracted using word- alignments induced with models that are unable to capture such reorderings, it is unlikely that …

Content truncated to 500 words