Entropy-based Pruning for Phrase-based Machine Translation

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 962–971, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Entropy-based Pruning for Phrase-based Machine Translation Wang Ling, Jo˜ao Grac¸a, Isabel Trancoso, Alan Black L2F Spoken Systems Lab, INESC-ID, Lisboa, Portugal Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA {wang.ling,joao.graca,isabel.trancoso}@inesc-id.pt awb@cs.cmu.edu Abstract Phrase-based machine translation models have shown to yield better translations than Word-based models, since phrase pairs en- code the contextual information that is needed for a more accurate translation. However, many phrase pairs do not encode any rele- vant context, which means that the transla- tion event encoded in that phrase pair is led by smaller translation events that are indepen- dent from each other, and can be found on smaller phrase pairs, with little or no loss in translation accuracy. In this work, we pro- pose a relative entropy model for translation models, that measures how likely a phrase pair encodes a translation event that is derivable using smaller translation events with similar probabilities. This model is then applied to phrase table pruning. Tests show that con- siderable amounts of phrase pairs can be ex- cluded, without much impact on the transla- tion quality. In fact, we show that better trans- lations can be obtained using our pruned mod- els, due to the compression of the search space during decoding. 1 Introduction Phrase-based Machine Translation Models (Koehn et al., 2003) model n-to-m translations of n source words to m target words, which are encoded in phrase pairs and stored in the translation model. This approach has an advantage over Word-based Translation Models (Brown et al., 1993), since trans- lating multiple source words allows the context for each source word to be considered during trans- lation. For instance, the translation of the En- glish word “in” by itself to Portuguese is not ob- vious, since we do not have any context for the word. This word can be translated in the con- text of “in (the box)” to “dentro”, or in the con- text of “in (China)” as “na”. In fact, the lexical entry for “in” has more than 10 good translations in Portuguese. Consequently, the lexical translation entry for Word-based models splits the probabilis- tic mass between different translations, leaving the choice based on context to the language model. On the other hand, in Phrase-based Models, we would have a phrase pair p(in the box, dentro da caixa) and p(in china, na china), where the words “in the box” and “in China” can be translated together to “dentro da caixa” and “na China”, which substan- tially reduces the ambiguity. In this case, both the translation and language models contribute to find the best translation based on the local context, which generally leads to better translations. However, not all words add the same amount of contextual information. Using the same example for “in”, if we add the context “(hid the key) in”, it is still not …

Content truncated to 500 words