Scalable Inference and Training of Context-Rich Syntactic Translation Models

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 961–968, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Scalable Inference and Training of Context-Rich Syntactic Translation Models Michel Galley*, Jonathan Graehl†, Kevin Knight†‡, Daniel Marcu†‡, Steve DeNeefe†, Wei Wang‡ and Ignacio Thayer† *Columbia University Dept. of Computer Science New York, NY 10027 galley@cs.columbia.edu, {graehl,knight,marcu,sdeneefe}@isi.edu, wwang@languageweaver.com, thayer@google.com †University of Southern California Information Sciences Institute Marina del Rey, CA 90292 ‡Language Weaver, Inc. 4640 Admiralty Way Marina del Rey, CA 90292 Abstract Statistical MT has made great progress in the last few years, but current translation models are weak on re-ordering and target language fluency. Syn- tactic approaches seek to remedy these problems. In this paper, we take the framework for acquir- ing multi-level syntactic translation rules of (Gal- ley et al., 2004) from aligned tree-string pairs, and present two main extensions of their approach: first, instead of merely computing a single derivation that minimally explains a sentence pair, we construct a large number of derivations that include contex- tually richer rules, and account for multiple inter- pretations of unaligned words. Second, we pro- pose probability estimates and a training procedure for weighting these rules. We contrast different approaches on real examples, show that our esti- mates based on multiple derivations favor phrasal re-orderings that are linguistically better motivated, and establish that our larger rules provide a 3.63 BLEU point increase over minimal rules. 1 Introduction While syntactic approaches seek to remedy word- ordering problems common to statistical machine translation (SMT) systems, many of the earlier models—particularly child re-ordering models— fail to account for human translation behavior. Galley et al. (2004) alleviate this modeling prob- lem and present a method for acquiring millions of syntactic transfer rules from bilingual corpora, which we review below. Here, we make the fol- lowing new contributions: (1) we show how to acquire larger rules that crucially condition on more syntactic context, and show how to com- pute multiple derivations for each training exam- ple, capturing both large and small rules, as well as multiple interpretations for unaligned words; (2) we develop probability models for these multi- level transfer rules, and give estimation methods for assigning probabilities to very large rule sets. We contrast our work with (Galley et al., 2004), highlight some severe limitations of probability estimates computed from single derivations, and demonstrate that it is critical to account for many derivations for each sentence pair. We also use real examples to show that our probability mod- els estimated from a large number of derivations favor phrasal re-orderings that are linguistically well motivated. An empirical evaluation against a state-of-the-art SMT system similar to (Och and Ney, 2004) indicates positive prospects. Finally, we show that our contextually richer rules provide a 3.63 BLEU point increase over those of (Galley et al., 2004). 2 Inferring syntactic transformations We assume we are given a source-language (e.g., French) sentence f, a target-language (e.g., En- glish) parse tree π, whose yield e is …

Content truncated to 500 words