A Markov Model of Machine Translation using Non-parametric Bayesian Inference

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 333–342, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics A Markov Model of Machine Translation using Non-parametric Bayesian Inference Yang Feng and Trevor Cohn Department of Computer Science The University of Sheffield Sheffield, United Kingdom yangfeng145@gmail.com and t.cohn@sheffield.ac.uk Abstract Most modern machine translation systems use phrase pairs as translation units, al- lowing for accurate modelling of phrase- internal translation and reordering. How- ever phrase-based approaches are much less able to model sentence level effects between different phrase-pairs. We pro- pose a new model to address this im- balance, based on a word-based Markov model of translation which generates tar- get translations left-to-right. Our model encodes word and phrase level phenom- ena by conditioning translation decisions on previous decisions and uses a hierar- chical Pitman-Yor Process prior to pro- vide dynamic adaptive smoothing. This mechanism implicitly supports not only traditional phrase pairs, but also gapping phrases which are non-consecutive in the source. Our experiments on Chinese to English and Arabic to English translation show consistent improvements over com- petitive baselines, of up to +3.4 BLEU. 1 Introduction Recent years have witnessed burgeoning develop- ment of statistical machine translation research, notably phrase-based (Koehn et al., 2003) and syntax-based approaches (Chiang, 2005; Galley et al., 2006; Liu et al., 2006). These approaches model sentence translation as a sequence of sim- ple translation decisions, such as the application of a phrase translation in phrase-based methods or a grammar rule in syntax-based approaches. In order to simplify modelling, most MT mod- els make an independence assumption, stating that the translation decisions in a derivation are in- dependent of one another. This conflicts with the intuition behind phrase-based MT, namely that translation decisions should be dependent on con- text. On one hand, the use of phrases can mem- orize local context and hence helps to generate better translation compared to word-based models (Brown et al., 1993; Och and Ney, 2003). On the other hand, this mechanism requires each phrase to be matched strictly and to be used as a whole, which precludes the use of discontinuous phrases and leads to poor generalisation to unseen data (where large phrases tend not to match). In this paper we propose a new model to drop the independence assumption, by instead mod- elling correlations between translation decisions, which we use to induce translation derivations from aligned sentences (akin to word alignment). We develop a Markov model over translation de- cisions, in which each decision is conditioned on previous n most recent decisions. Our approach employs a sophisticated Bayesian non-parametric prior, namely the hierarchical Pitman-Yor Process (Teh, 2006; Teh et al., 2006) to represent back- off from larger to smaller contexts. As a result, we need only use very simple translation units – primarily single words, but can still describe complex multi-word units through correlations be- tween their component translation decisions. We further decompose the process of generating each target word into component factors: finishing the …

Content truncated to 500 words