Factored Markov Translation with Robust Modeling

Proceedings of the Eighteenth Conference on Computational Language Learning, pages 151–159, Baltimore, Maryland USA, June 26-27 2014. c⃝2014 Association for Computational Linguistics Factored Markov Translation with Robust Modeling Yang Feng† Trevor Cohn‡ Xinkai Du † Information Sciences Institue ‡ Computing and Information Systems Computer Science Department The University of Melbourne University of Southern California VIC 3010 Australia {yangfeng145, xinkaid}@gmail.com t.cohn@unimelb.edu.au Abstract Phrase-based translation models usually memorize local translation literally and make independent assumption between phrases which makes it neither generalize well on unseen data nor model sentence- level effects between phrases. In this pa- per we present a new method to model correlations between phrases as a Markov model and meanwhile employ a robust smoothing strategy to provide better gen- eralization. This method defines a re- cursive estimation process and backs off in parallel paths to infer richer structures. Our evaluation shows an 1.1–3.2% BLEU improvement over competitive baselines for Chinese-English and Arabic-English translation. 1 Introduction Phrase-based methods to machine translation (Koehn et al., 2003; Koehn et al., 2007) have dras- tically improved beyond word-based approaches, primarily by using phrase-pairs as translation units, which can memorize local lexical con- text and reordering patterns. However, this lit- eral memorization mechanism makes it general- ize poorly to unseen data. Moreover, phrase-based models make an independent assumption, stating that the application of phrases in a derivation is in- dependent to each other which conflicts with the underlying truth that the translation decisions of phrases should be dependent on context. There are some work aiming to solve the two problems. Feng and Cohn (2013) propose a word-based Markov model to integrate translation and reordering into one model and use the so- phisticated hierarchical Pitman-Yor process which backs off from larger to smaller context to pro- vide dynamic adaptive smoothing. This model shows good generalization to unseen data while it uses words as the translation unit which can- not handle multiple-to-multiple links in real word alignments. Durrani et al. (2011) and Durrani et al. (2013) propose an operation sequence model (OSM) which models correlations between mini- mal translation units (MTUs) and evaluates proba- bilities with modified Kneser-Ney smoothing. On one hand the use of MTUs can help retain the multiple-to-multiple alignments, on the other hand its definition of operations where source words and target words are bundled into one operation makes it subjected to sparsity. The common fea- ture of the above two methods is they both back off in one fixed path by dropping least recent events first which precludes some useful structures. For the segment pairs <bˇa t¯a kˇaol`v j`ınq`u, take it into account> in Figure 1, the more common structure is <bˇa ... kˇaol`v j`ınq`u, take ... into account>. If we always drop the least recent events first, then we can only learn the pattern <... t¯a kˇaol`v j`ınq`u, ... it into account>. On these grounds, we propose a method with new definition of correlations and more robust probability modeling. This method defines a Markov model over correlations between minimal phrases where …

Content truncated to 500 words