The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015

Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 126–133, Lisboa, Portugal, 17-18 September 2015. c⃝2015 Association for Computational Linguistics. The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015 Barry Haddow1, Matthias Huck1, Alexandra Birch1, Nikolay Bogoychev1, Philipp Koehn1,2 1School of Informatics, University of Edinburgh 2Center for Speech and Language Processing, The Johns Hopkins University a.birch@ed.ac.uk {nbogoych,bhaddow,mhuck}@inf.ed.ac.uk phi@jhu.edu Abstract This paper describes the submission of the University of Edinburgh and the Johns Hopkins University for the shared transla- tion task of the EMNLP 2015 Tenth Work- shop on Statistical Machine Translation (WMT 2015). We set up phrase-based sta- tistical machine translation systems for all ten language pairs of this year’s evaluation campaign, which are English paired with Czech, Finnish, French, German, and Rus- sian in both translation directions. Novel research directions we investigated include: neural network language mod- els and bilingual neural network language models, a comprehensive use of word classes, and sparse lexicalized reordering features. 1 Introduction The Edinburgh/JHU phrase-based translation sys- tems for our participation in the WMT 2015 shared translation task1 are based on the open source Moses toolkit (Koehn et al., 2007). We built upon Edinburgh’s strong baselines from WMT submissions in previous years (Durrani et al., 2014a) as well as our recent research within the framework of other evaluation campaigns and projects such as IWSLT2 and EU-BRIDGE3 (Birch et al., 2014; Freitag et al., 2014a; Freitag et al., 2014b). We first discuss novel features that we in- tegrated into our systems for the 2015 Edin- burgh/JHU submission. Next we give a gen- eral system overview with details on our train- ing pipeline and decoder configuration. We fi- nally present empirical results for the individual language pairs and translation directions. 1http://www.statmt.org/wmt15/ 2http://workshop2014.iwslt.org 3http://www.eu-bridge.eu 2 Novel Methods 2.1 Neural Network LM with NPLM For some language pairs (notably French↔English and Finnish↔English) we ex- perimented with feed-forward neural network lan- guage models using the NPLM toolkit (Vaswani et al., 2013). This toolkit enables such language models to be trained efficiently on large datasets, and provides a querying API which is fast enough to be used during decoding. NPLM is fully inte- grated into Moses, including appropriate wrapper scripts for training the language models within the Moses experiment management system. 2.2 Bilingual Neural Network LM We also experimented with our re-implementation of the “joint” model by Devlin et al. (2014). Re- ferred to as bilingual LM in Moses, this was pre- viously employed in the Edinburgh IWSLT system submissions, although with limited success (Birch et al., 2014). The idea of the bilingual LM is quite straight- forward. We define a language model where each target token is conditioned on the previous (n−1) target tokens (as in a standard n-gram language model) as well as its aligned source token, and a window of m tokens on either side of the aligned source token. At training time, the aligned source token is found from the automatic alignment, and at test time the alignment is supplied by the de- coder. The …

Content truncated to 500 words