A Systematic Comparison of Phrase Table Pruning Techniques

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 972–983, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics A Systematic Comparison of Phrase Table Pruning Techniques Richard Zens and Daisy Stanton and Peng Xu Google Inc. {zens,daisy,xp}@google.com Abstract When trained on very large parallel corpora, the phrase table component of a machine translation system grows to consume vast computational resources. In this paper, we in- troduce a novel pruning criterion that places phrase table pruning on a sound theoretical foundation. Systematic experiments on four language pairs under various data conditions show that our principled approach is superior to existing ad hoc pruning methods. 1 Introduction Over the last years, statistical machine translation has become the dominant approach to machine translation. This is not only due to improved mod- eling, but also due to a significant increase in the availability of monolingual and bilingual data. Here are just two examples of very large data resources that are publicly available: • The Google Web 1T 5-gram corpus available from the Linguistic Data Consortium consist- ing of the 5-gram counts of about one trillion words of web data.1 • The 109-French-English bilingual corpus with about one billion tokens from the Workshop on Statistical Machine Translation (WMT).2 These enormous data sets yield translation models that are expensive to store and process. Even with 1LDC catalog No. LDC2006T13 2http://www.statmt.org/wmt11/translation-task.html modern computers, these large models lead to a long experiment cycle that hinders progress. The situa- tion is even more severe if computational resources are limited, for instance when translating on hand- held devices. Then, reducing the model size is of the utmost importance. The most resource-intensive components of a sta- tistical machine translation system are the language model and the phrase table. Recently, compact rep- resentations of the language model have attracted the attention of the research community, for instance in Talbot and Osborne (2007), Brants et al. (2007), Pauls and Klein (2011) or Heafield (2011), to name a few. In this paper, we address the other problem of any statistical machine translation system: large phrase tables. Johnson et al. (2007) has shown that large por- tions of the phrase table can be removed without loss in translation quality. This motivated us to perform a systematic comparison of different pruning meth- ods. However, we found that many existing methods employ ad-hoc heuristics without theoretical foun- dation. The pruning criterion introduced in this work is inspired by the very successful and still state-of-the- art language model pruning criterion based on en- tropy measures (Stolcke, 1998). We motivate its derivation by stating the desiderata for a good phrase table pruning criterion: • Soundness: The criterion should optimize some well-understood information-theoretic measure of translation model quality. 972 • Efficiency: Pruning should be fast, i. e., run lin- early in the size of the phrase table. • Self-containedness: As a practical considera- tion, we want to prune phrases from an existing phrase table. This means pruning …

Content truncated to 500 words