Bayesian Learning of Phrasal Tree-to-String Templates

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1308–1317, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Bayesian Learning of Phrasal Tree-to-String Templates Ding Liu and Daniel Gildea Department of Computer Science University of Rochester Rochester, NY 14627 {dliu, gildea}@cs.rochester.edu Abstract We examine the problem of overcoming noisy word-level alignments when learn- ing tree-to-string translation rules. Our approach introduces new rules, and re- estimates rule probabilities using EM. The major obstacles to this approach are the very reasons that word-alignments are used for rule extraction: the huge space of possible rules, as well as controlling overfitting. By carefully controlling which portions of the original alignments are re- analyzed, and by using Bayesian infer- ence during re-analysis, we show signifi- cant improvement over the baseline rules extracted from word-level alignments. 1 Introduction Non-parametric Bayesian methods have been suc- cessfully applied to directly learn phrase pairs from a bilingual corpus with little or no depen- dence on word alignments (Blunsom et al., 2008; DeNero et al., 2008). Because such approaches di- rectly learn a generative model over phrase pairs, they are theoretically preferable to the standard heuristics for extracting the phrase pairs from the many-to-one word-level alignments produced by the IBM series models (Brown et al., 1993) or the Hidden Markov Model (HMM) (Vogel et al., 1996). We wish to apply this direct, Bayesian ap- proach to learn better translation rules for syntax- based statistical MT (SSMT), by which we specif- ically refer to MT systems using Tree-to-String (TTS) translation templates derived from syntax trees (Liu et al., 2006; Huang et al., 2006; Gal- ley et al., 2006; May and Knight, 2007), as op- posed to formally syntactic systems such as Hi- ero (Chiang, 2007). The stumbling block pre- venting us from taking this approach is the ex- tremely large space of possible TTS templates when no word alignments are given. Given a sen- tence pair and syntax tree over one side, there are an exponential number of potential TTS tem- plates and a polynomial number of phrase pairs. In this paper, we explore methods for restricting the space of possible TTS templates under con- sideration, while still allowing good templates to emerge directly from the data as much as possible. We find an improvement in translation accuracy through, first, using constraints to limit the number of new templates, second, using Bayesian methods to limit which of these new templates are favored when re-analyzing the training data with EM, and, third, experimenting with different renormaliza- tion techniques for the EM re-analysis. We introduce two constraints to limit the num- ber of TTS templates that we extract directly from tree/string pairs without using word alignments. The first constraint is to limit direct TTS tem- plate extraction to the part of the corpus where word alignment tools such as GIZA++ do poorly. There is no reason not to re-use the good align- ments from GIZA++, which holds a very compet- itive baseline performance. As already mentioned, the noisy alignments …

Content truncated to 500 words