Deciphering Foreign Language

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 12–21, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Deciphering Foreign Language Sujith Ravi and Kevin Knight University of Southern California Information Sciences Institute Marina del Rey, California 90292 {sravi,knight}@isi.edu Abstract In this work, we tackle the task of ma- chine translation (MT) without parallel train- ing data. We frame the MT problem as a de- cipherment task, treating the foreign text as a cipher for English and present novel meth- ods for training translation models from non- parallel text. 1 Introduction Bilingual corpora are a staple of statistical machine translation (SMT) research. From these corpora, we estimate translation model parameters: word- to-word translation tables, fertilities, distortion pa- rameters, phrase tables, syntactic transformations, etc. Starting with the classic IBM work (Brown et al., 1993), training has been viewed as a maximiza- tion problem involving hidden word alignments (a) that are assumed to underlie observed sentence pairs (e, f): arg max θ � e,f Pθ(f|e) (1) = arg max θ � e,f � a Pθ(f, a|e) (2) Brown et al. (1993) give various formulas that boil Pθ(f, a|e) down to the specific parameters to be es- timated. Of course, for many language pairs and domains, parallel data is not available. In this paper, we address the problem of learning a full transla- tion model from non-parallel data, and we use the learned model to translate new foreign strings. As successful work develops along this line, we expect more domains and language pairs to be conquered by SMT. How can we learn a translation model from non- parallel data? Intuitively, we try to construct trans- lation model tables which, when applied to ob- served foreign text, consistently yield sensible En- glish. This is essentially the same approach taken by cryptanalysts and epigraphers when they deal with source texts. In our case, we observe a large number of foreign strings f, and we apply maximum likelihood train- ing: arg max θ � f Pθ(f) (3) Following Weaver (1955), we imagine that this cor- pus of foreign strings “is really written in English, but has been coded in some strange symbols,” thus: arg max θ � f � e P(e) · Pθ(f|e) (4) The variable e ranges over all possible English strings, and P(e) is a language model built from large amounts of English text that is unrelated to the foreign strings. Re-writing for hidden alignments, we get: arg max θ � f � e P(e) · � a Pθ(f, a|e) (5) Note that this formula has the same free Pθ(f, a|e) parameters as expression (2). We seek to manipulate these parameters in order to learn the 12 same full translation model. We note that for each f, not only is the alignment a still hidden, but now the English translation e is hidden as well. A language model P(e) is typically used in SMT decoding (Koehn, 2009), but here P(e) actually plays a central role in training translation …

Content truncated to 500 words