Word Translation Prediction for Morphologically Rich Languages with Bilingual Neural Networks

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1676–1688, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Word Translation Prediction for Morphologically Rich Languages with Bilingual Neural Networks Ke Tran Arianna Bisazza Christof Monz Informatics Institute, University of Amsterdam Science Park 904, 1098 XH Amsterdam, The Netherlands {m.k.tran,a.bisazza,c.monz}@uva.nl Abstract Translating into morphologically rich lan- guages is a particularly difficult problem in machine translation due to the high de- gree of inflectional ambiguity in the tar- get language, often only poorly captured by existing word translation models. We present a general approach that exploits source-side contexts of foreign words to improve translation prediction accuracy. Our approach is based on a probabilistic neural network which does not require lin- guistic annotation nor manual feature en- gineering. We report significant improve- ments in word translation prediction accu- racy for three morphologically rich target languages. In addition, preliminary results for integrating our approach into a large- scale English-Russian statistical machine translation system show small but statisti- cally significant improvements in transla- tion quality. 1 Introduction The ability to make context-sensitive translation decisions is one of the major strengths of phrase- based SMT (PSMT). However, the way PSMT ex- ploits source-language context has several limita- tions as pointed out, for instance, by Quirk and Menezes (2006) and Durrani et al. (2013). First, the amount of context used to translate a given input word depends on the phrase segmentation, with hypotheses resulting from different segmen- tations competing with one another. Another issue is that, given a phrase segmentation, each source phrase is translated independently from the oth- ers, which can be problematic especially for short phrases. As a result, the predictive translation of a source phrase does not access useful linguistic clues in the source sentence that are outside of the scope of the phrase. Lexical weighting tackles the problem of un- reliable phrase probabilities, typically associated with long phrases, but does not alleviate the prob- lem of context segmentation. An important share of the translation selection task is then left to the language model (LM), which is certainly very ef- fective but can only leverage target language con- text. Moreover, decisions that are taken at early decoding stages—such as the common practice of retaining only top n translation options for each source span—depend only on the translation models and on the target context available in the phrase. Source context based translation models (Gim- pel and Smith, 2008; Mauser et al., 2009; Jeong et al., 2010; Haque et al., 2011) naturally ad- dress these limitations. These models can ex- ploit a boundless context of the input text, but they assume that target words can be predicted in- dependently from each other, which makes them easy to integrate into state-of-the-art PSMT sys- tems. Even though the independence assump- tion is made on the target side, these models have shown the benefits of utilizing source context, es- pecially in translating into morphologically rich languages. One drawback of previous research on …

Content truncated to 500 words