Multi-Source Neural Translation

Proceedings of NAACL-HLT 2016, pages 30–34, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Multi-Source Neural Translation Barret Zoph and Kevin Knight Information Sciences Institute Department of Computer Science University of Southern California {zoph,knight}@isi.edu Abstract We build a multi-source machine translation model and train it to maximize the probabil- ity of a target English string given French and German sources. Using the neural encoder- decoder framework, we explore several com- bination methods and report up to +4.8 Bleu increases on top of a very strong attention- based neural translation model. 1 Introduction Kay (2000) points out that if a document is trans- lated once, it is likely to be translated again and again into other languages. This gives rise to an in- teresting idea: a human does the first translation by hand, then turns the rest over to machine translation (MT). The translation system now has two strings as input, which can reduce ambiguity via “triangu- lation” (Kay’s term). For example, the normally ambiguous English word “bank” may be more eas- ily translated into French in the presence of a sec- ond, German input string containing the word “Flus- sufer” (river bank). Och and Ney (2001) describe such a multi-source MT system. They first train separate bilingual MT systems F→E, G→E, etc. At runtime, they sep- arately translate input strings f and g into candi- date target strings e1 and e2, then select the best one of the two. A typical selection factor is the prod- uct of the system scores. Schwartz (2008) revisits such factors in the context of log-linear models and Bleu score, while Max et al. (2010) re-rank F→E n-best lists using n-gram precision with respect to G→E translations. Callison-Burch (2002) exploits hypothesis selection in multi-source MT to expand available corpora, via co-training. Others use system combination techniques to merge hypotheses at the word level, creating the ability to synthesize new translations outside those proposed by the single-source translators. These methods include confusion networks (Matusov et al., 2006; Schroeder et al., 2009), source-side string combination (Schroeder et al., 2009), and median strings (Gonz´alez-Rubio and Casacuberta, 2010). The above work all relies on base MT systems trained on bilingual data, using traditional meth- ods. This follows early work in sentence align- ment (Gale and Church, 1993) and word alignment (Simard, 1999), which exploited trilingual text, but did not build trilingual models. Previous authors possibly considered a three-dimensional translation table t(e|f, g) to be prohibitive. In this paper, by contrast, we train a P(e|f, g) model directly on trilingual data, and we use that model to decode an (f, g) pair simultaneously. We view this as a kind of multi-tape transduction (Elgot and Mezei, 1965; Kaplan and Kay, 1994; Deri and Knight, 2015) with two input tapes and one output tape. Our contributions are as follows: • We train a P(e|f, g) model directly on trilin- gual data, and we use it to decode a new source string pair (f, g) into target string e. • …

Content truncated to 500 words