Bayesian Extraction of Minimal SCFG Rules for Hierarchical Phrase-based Translation

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 533–541, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics Bayesian Extraction of Minimal SCFG Rules for Hierarchical Phrase-based Translation Baskaran Sankaran Simon Fraser University Burnaby BC, Canada baskaran@cs.sfu.ca Gholamreza Haffari Monash University Melbourne, Australia reza@monash.edu Anoop Sarkar Simon Fraser University Burnaby BC, Canada anoop@cs.sfu.ca Abstract We present a novel approach for extracting a minimal synchronous context-free grammar (SCFG) for Hiero-style statistical machine translation using a non-parametric Bayesian framework. Our approach is designed to ex- tract rules that are licensed by the word align- ments and heuristically extracted phrase pairs. Our Bayesian model limits the number of SCFG rules extracted, by sampling from the space of all possible hierarchical rules; addi- tionally our informed prior based on the lex- ical alignment probabilities biases the gram- mar to extract high quality rules leading to im- proved generalization and the automatic iden- tification of commonly re-used rules. We show that our Bayesian model is able to ex- tract minimal set of hierarchical phrase rules without impacting the translation quality as measured by the BLEU score. 1 Introduction Hierarchical phrase-based (Hiero) machine transla- tion (Chiang, 2007) has attracted significant interest within the Machine Translation community. It ex- tends phrase-based translation by automatically in- ferring a synchronous grammar from an aligned bi- text. The synchronous context-free grammar links non-terminals in source and target languages. De- coding in such systems employ a modified CKY- parser that is integrated with a language model. The primary advantage of Hiero-style systems lie in their unsupervised model of syntax for transla- tion: allowing long-distance reordering and cap- turing certain syntactic constructions, particularly those that involve discontiguous phrases. It has been demonstrated to be a successful framework with comparable performance with other statisti- cal frameworks and suitable for large-scale cor- pora (Zollmann et al., 2008). However, one of the major difficulties in Hiero-style systems has been on learning a concise and general synchronous gram- mar from the bitext. While most of the research in Hiero-style sys- tems is focused on the improving the decoder, and in particular the link to the language model, compar- atively few papers have considered the inference of the probabilistic SCFG from the word alignments. A majority of the systems employ the classic rule- extraction algorithm (Chiang, 2007) which extracts rules by replacing possible sub-spans (permitted by the word alignments) with a non-terminal and then using relative frequencies to estimate the probabilis- tic synchronous context-free grammar. One of the issues in building Hiero-style systems is in manag- ing the size of the synchronous grammar. The origi- nal approach extracts a larger number of rules when compared to a phrase-based system on the same data leading to practical issues in terms of memory re- quirements and decoding speed. Extremely large Hiero phrase tables may also lead to statistical issues, where the probability mass has to be shared by more rules: the probability p(e|f) has to be shared by all the rules having the same source …

Content truncated to 500 words