Preference Grammars: Softening Syntactic Constraints to Improve Statistical Machine Translation

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 236–244, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Preference Grammars: Softening Syntactic Constraints to Improve Statistical Machine Translation Ashish Venugopal Andreas Zollmann Noah A. Smith Stephan Vogel Language Technologies Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA {ashishv,zollmann,nasmith,vogel}@cs.cmu.edu Abstract We propose a novel probabilistic syn- choronous context-free grammar formalism for statistical machine translation, in which syntactic nonterminal labels are represented as “soft” preferences rather than as “hard” matching constraints. This formalism allows us to efficiently score unlabeled synchronous derivations without forgoing traditional syntactic constraints. Using this score as a feature in a log-linear model, we are able to approximate the selection of the most likely unlabeled derivation. This helps reduce fragmentation of probability across differently labeled derivations of the same translation. It also allows the importance of syntactic preferences to be learned alongside other features (e.g., the language model) and for particular labeling procedures. We show improvements in translation quality on small and medium sized Chinese-to-English translation tasks. 1 Introduction Probabilistic synchronous context-free grammars (PSCFGs) define weighted production rules that are automatically learned from parallel training data. As in classical CFGs, these rules make use of nontermi- nal symbols to generalize beyond lexical modeling of sentences. In MT, this permits translation and re- ordering to be conditioned on more abstract notions of context. For example, VP → ne VB1 pas # do not VB1 represents the discontiguous translation of the French words “ne” and “pas” to “do not”, in the con- text of the labeled nonterminal symbol “VB” (rep- resenting syntactic category “verb”). Translation with PSCFGs is typically expressed as the problem of finding the maximum-weighted derivation consis- tent with the source sentence, where the scores are defined (at least in part) by R-valued weights asso- ciated with the rules. A PSCFG derivation is a syn- chronous parse tree. Defining the translation function as finding the best derivation has the unfortunate side effect of forcing differently-derived versions of the same tar- get sentence to compete with each other. In other words, the true score of each translation is “frag- mented” across many derivations, so that each trans- lation’s most probable derivation is the only one that matters. The more Bayesian approach of finding the most probable translation (integrating out the derivations) instantiates an NP-hard inference prob- lem even for simple word-based models (Knight, 1999); for grammar-based translation it is known as the consensus problem (Casacuberta and de la Higuera, 2000; Sima’an, 2002). With weights interpreted as probabilities, the maximum-weighted derivation is the maximum a posteriori (MAP) derivation: ˆe ← argmax e max d p(e, d | f) where f is the source sentence, e ranges over tar- get sentences, and d ranges over PSCFG deriva- tions (synchronous trees). This is often described as an approximation to the most probable transla- tion, argmaxe � d p(e, d | f). In this paper, we will describe a technique that aims …

Content truncated to 500 words