Learning Probabilistic Synchronous CFGs for Phrase-Based Translation

Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 117–125, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics Learning Probabilistic Synchronous CFGs for Phrase-based Translation Markos Mylonakis ILLC University of Amsterdam m.mylonakis@uva.nl Khalil Sima’an ILLC University of Amsterdam k.simaan@uva.nl Abstract Probabilistic phrase-based synchronous grammars are now considered promis- ing devices for statistical machine transla- tion because they can express reordering phenomena between pairs of languages. Learning these hierarchical, probabilistic devices from parallel corpora constitutes a major challenge, because of multiple la- tent model variables as well as the risk of data overfitting. This paper presents an effective method for learning a family of particular interest to MT, binary Syn- chronous Context-Free Grammars with in- verted/monotone orientation (a.k.a. Bi- nary ITG). A second contribution con- cerns devising a lexicalized phrase re- ordering mechanism that has complimen- tary strengths to Chiang’s model. The latter conditions reordering decisions on the surrounding lexical context of phrases, whereas our mechanism works with the lexical content of phrase pairs (akin to standard phrase-based systems). Surpris- ingly, our experiments on French-English data show that our learning method ap- plied to far simpler models exhibits per- formance indistinguishable from the Hiero system. 1 Introduction A fundamental problem in phrase-based machine translation concerns the learning of a probabilistic synchronous context-free grammar (SCFG) over phrase pairs from an input parallel corpus. Chi- ang’s Hiero system (Chiang, 2007) exemplifies the gains to be had by combining phrase-based translation (Och and Ney, 2004) with the hierar- chical reordering capabilities of SCFGs, particu- larly originating from Binary Inversion Transduc- tion Grammars (BITG) (Wu, 1997). Yet, exist- ing empirical work is largely based on successful heuristic techniques, and the learning of Hiero-like BITG/SCFG remains an unsolved problem, The difficulty of this problem stems from the need for simultaneously learning of two kinds of preferences (see Fig.1) (1) lexical translation prob- abilities (P(⟨e, f⟩ | X)) of source (f) and target (e) phrase pairs, and (2) phrase reordering prefer- ences of a target string relative to a source string, expressed in synchronous productions probabil- ities (for monotone or switching productions). Theoretically speaking, both kinds of preferences may involve latent structure relative to the paral- lel corpus. The mapping between source-target sentence pairs can be expressed in terms of la- tent phrase segmentations and latent word/phrase- alignments, and the hierarchical phrase reorder- ing can be expressed in terms of latent binary synchronous hierarchical structures (cf. Fig. 1). But each of these three kinds of latent structures may be made explicit using external resources: word-alignment could be considered solved us- ing Giza++ (Och and Ney, 2003)), phrase pairs can be obtained from these word-alignments (Och and Ney, 2004), and the hierarchical synchronous structure can be grown over source/target linguis- tic syntactic trees output by an existing parser. The Joint Phrase Translation Model (Marcu and Wong, 2002) constitutes a specific case, albeit without the hierarchical, synchronous reordering Start S → X 1 / X 1 (1) Monotone X → X 1 X 2 /X …

Content truncated to 500 words