An Infinite Hierarchical Bayesian Model of Phrasal Translation

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 780–790, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics An Infinite Hierarchical Bayesian Model of Phrasal Translation Trevor Cohn Department of Computer Science The University of Sheffield Sheffield, United Kingdom t.cohn@sheffield.ac.uk Gholamreza Haffari Faculty of Information Technology Monash University Clayton, Australia reza@monash.edu Abstract Modern phrase-based machine translation systems make extensive use of word- based translation models for inducing alignments from parallel corpora. This is problematic, as the systems are inca- pable of accurately modelling many trans- lation phenomena that do not decompose into word-for-word translation. This pa- per presents a novel method for induc- ing phrase-based translation units directly from parallel data, which we frame as learning an inverse transduction grammar (ITG) using a recursive Bayesian prior. Overall this leads to a model which learns translations of entire sentences, while also learning their decomposition into smaller units (phrase-pairs) recursively, terminat- ing at word translations. Our experiments on Arabic, Urdu and Farsi to English demonstrate improvements over competi- tive baseline systems. 1 Introduction The phrase-based approach (Koehn et al., 2003) to machine translation (MT) has transformed MT from a narrow research topic into a truly useful technology to end users. Leading translation sys- tems (Chiang, 2007; Koehn et al., 2007; Marcu et al., 2006) all use some kind of multi-word transla- tion unit, which allows translations to be produced from large canned units of text from the training corpus. Larger phrases allow for the lexical con- text to be considered in choosing the translation, and also limit the number of reordering decisions required to produce a full translation. Word-based translation models (Brown et al., 1993) remain central to phrase-based model train- ing, where they are used to infer word-level align- ments from sentence aligned parallel data, from which phrasal translation units are extracted us- ing a heuristic. Although this approach demon- strably works, it suffers from a number of short- comings. Firstly, many phrase-based phenomena which do not decompose into word translations (e.g., idioms) will be missed, as the underlying word-based alignment model is unlikely to pro- pose the correct alignments. Secondly, the rela- tionship between different phrase-pairs is not con- sidered, such as between single word translations and larger multi-word phrase-pairs or where one large phrase-pair subsumes another. This paper develops a phrase-based translation model which aims to address the above short- comings of the phrase-based translation pipeline. Specifically, we formulate translation using in- verse transduction grammar (ITG), and seek to learn an ITG from parallel corpora. The novelty of our approach is that we develop a Bayesian prior over the grammar, such that a nontermi- nal becomes a ‘cache’ learning each production and its complete yield, which in turn is recur- sively composed of its child constituents. This is closely related to adaptor grammars (Johnson et al., 2007a), which also generate full tree rewrites in a monolingual setting. Our model learns trans- lations of entire sentences while also learning their decomposition into smaller units …

Content truncated to 500 words