Toward Statistical Machine Translation without Parallel Corpora

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 130–140, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Toward Statistical Machine Translation without Parallel Corpora Alexandre Klementiev Ann Irvine Chris Callison-Burch David Yarowsky Center for Language and Speech Processing Johns Hopkins University Abstract We estimate the parameters of a phrase- based statistical machine translation sys- tem from monolingual corpora instead of a bilingual parallel corpus. We extend exist- ing research on bilingual lexicon induction to estimate both lexical and phrasal trans- lation probabilities for MT-scale phrase- tables. We propose a novel algorithm to es- timate reordering probabilities from mono- lingual data. We report translation results for an end-to-end translation system us- ing these monolingual features alone. Our method only requires monolingual corpora in source and target languages, a small bilingual dictionary, and a small bitext for tuning feature weights. In this paper, we ex- amine an idealization where a phrase-table is given. We examine the degradation in translation performance when bilingually estimated translation probabilities are re- moved and show that 80%+ of the loss can be recovered with monolingually estimated features alone. We further show that our monolingual features add 1.5 BLEU points when combined with standard bilingually estimated phrase table features. 1 Introduction The parameters of statistical models of transla- tion are typically estimated from large bilingual parallel corpora (Brown et al., 1993). However, these resources are not available for most lan- guage pairs, and they are expensive to produce in quantities sufficient for building a good transla- tion system (Germann, 2001). We attempt an en- tirely different approach; we use cheap and plen- tiful monolingual resources to induce an end-to- end statistical machine translation system. In par- ticular, we extend the long line of work on in- ducing translation lexicons (beginning with Rapp (1995)) and propose to use multiple independent cues present in monolingual texts to estimate lex- ical and phrasal translation probabilities for large, MT-scale phrase-tables. We then introduce a novel algorithm to estimate reordering features from monolingual data alone, and we report the performance of a phrase-based statistical model (Koehn et al., 2003) estimated using these mono- lingual features. Most of the prior work on lexicon induction is motivated by the idea that it could be applied to machine translation but stops short of actu- ally doing so. Lexicon induction holds the po- tential to create machine translation systems for languages which do not have extensive parallel corpora. Training would only require two large monolingual corpora and a small bilingual dictio- nary, if one is available. The idea is that intrin- sic properties of monolingual data (possibly along with a handful of bilingual pairs to act as exam- ple mappings) can provide independent but infor- mative cues to learn translations because words (and phrases) behave similarly across languages. This work is the first attempt to extend and apply these ideas to an end-to-end machine translation pipeline. While we make an explicit assumption that a table …

Content truncated to 500 words