Going Beyond Word Cooccurrences in Global Lexical Selection for Statistical Machine Translation using a Multilayer Perceptron

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 658–666, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Going Beyond Word Cooccurrences in Global Lexical Selection for Statistical Machine Translation using a Multilayer Perceptron Alexandre Patry KeaText 845 Blvd. D´ecarie, Suite 202 Montr´eal, Canada H4L 3L7 alexandre.patry@keatext.com Philippe Langlais DIRO/RALI Universit´e de Montr´eal Montr´eal, Canada H3C 3J7 felipe@iro.umontreal.ca Abstract Phrase-based statistical machine transla- tion (PBSMT) decoders translate source sentences one phrase at a time using strong independence assumptions over the source phrases. Translation table scores are typ- ically independent of context, language model scores depend on a few words sur- rounding the target phrase and distortion models do not influence directly the choice of target phrases. In this work, we propose to condition the selection of each target word on the whole source sentence using a multilayer percep- tron (MLP). Our interest in MLP lies in their hidden layer which encodes source sentences in a representation that is not di- rectly tied to the notion of word. We evaluated our approach on an English to French translation task. Our MLP model was able to improve BLEU scores over a standard PBSMT system. 1 Introduction Phrase-based statistical machine translation sys- tems translate source sentences step by step, start- ing with an empty sentence and ending when all source words have been translated (Koehn et al., 2003). At each step, an untranslated phrase is se- lected and one of its translation is appended at the end of the translation. In this work, we are interested in the selec- tion of a target phrase to translate a given source phrase. This selection is usually guided by three families of models. Translation models evaluate the intrinsic quality of a given phrase pair using evidences such as cooccurrence statistics between source and target words or phrases. These models always compute the same scores for a given pair of phrases, wherever it is used. A second family are language models, which evaluate the likeliness of target n-grams1 independently from the source sentence. A third family are distortion models, whose main purpose is to evaluate the likelihood of phrase reorderings. All these models only consider a fraction of the information available at a time. Figure 1 presents a partial translation that could be encountered by a decoder where these fractions of information are not enough to make a good decision. In this ex- ample, the decoder has already translated all but the last source word and it must decide if plant is translated by usine (a building) or plante (the botanic sense). As the position of the source word is fixed, the distortion model will not be of any help. If the system uses a 3-gram, only the last two words (de cette) of the partial hypothesis will be considered. Finally, the translation model will score the two options considering only plant. Therefore, the word leaves is never considered and the final decision largely depends on whether the training corpus is more …

Content truncated to 500 words