Smooth Bilingual N-Gram Translation

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 430–438, Prague, June 2007. c⃝2007 Association for Computational Linguistics Smooth Bilingual N-gram Translation Holger Schwenk Marta R. Costa-juss`a and Jos´e A.R. Fonollosa LIMSI-CNRS, BP 133 91403 Orsay cedex, FRANCE schwenk@lismi.fr UPC - TALP Barcelona 08034, Spain {mruiz,adrian}@gps.tsc.upc.edu Abstract We address the problem of smoothing trans- lation probabilities in a bilingual N-gram- based statistical machine translation system. It is proposed to project the bilingual tuples onto a continuous space and to estimate the translation probabilities in this representa- tion. A neural network is used to perform the projection and the probability estimation. Smoothing probabilities is most important for tasks with a limited amount of training material. We consider here the BTEC task of the 2006 IWSLT evaluation. Improve- ments in all official automatic measures are reported when translating from Italian to En- glish. Using a continuous space model for the translation model and the target language model, an improvement of 1.5 BLEU on the test data is observed. 1 Introduction The goal of statistical machine translation (SMT) is to produce a target sentence e from a source sen- tence f. Among all possible target language sen- tences the one with the highest probability is chosen: e∗ = arg max e Pr(e|f) = arg max e Pr(f|e) Pr(e) where Pr(f|e) is the translation model and Pr(e) is the target language model. This approach is usually referred to as the noisy source-channel ap- proach in statistical machine translation (Brown et al., 1993). During the last few years, the use of context in SMT systems has provided great improvements in translation. SMT has evolved from the origi- nal word-based approach to phrase-based translation systems (Och et al., 1999; Koehn et al., 2003). A phrase is defined as a group of source words ˜f that should be translated together into a group of target words ˜e. The translation model in phrase-based sys- tems includes the phrase translation probabilities in both directions, i.e. P(˜e|˜f) and P(˜f|˜e). The use of a maximum entropy approach simpli- fies the introduction of several additional models ex- plaining the translation process : e∗ = arg max p(e|f) = arg max e {exp( � i λihi(e, f))} (1) The feature functions hi are the system models and the λi weights are typically optimized to maximize a scoring function on a development set (Och and Ney, 2002). The phrase translation probabilities P(˜e|˜f) and P(˜f|˜e) are usually obtained using relative frequency estimates. Statistical learning theory, however, tells us that relative frequency estimates have several drawbacks, in particular high variance and low bias. Phrase tables may contain several millions of en- tries, most of which appear only once or twice, which means that we are confronted with a data sparseness problem. Surprisingly, there seems to be little work addressing the issue of smoothing of the phrase table probabilities. On the other hand, smoothing of relative fre- quency estimates was extensively investigated in the 430 area of …

Content truncated to 500 words