Phrasetable Smoothing for Statistical Machine Translation

Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 53–61, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Phrasetable Smoothing for Statistical Machine Translation George Foster and Roland Kuhn and Howard Johnson National Research Council Canada Ottawa, Ontario, Canada firstname.lastname@nrc.gc.ca Abstract We discuss different strategies for smooth- ing the phrasetable in Statistical MT, and give results over a range of translation set- tings. We show that any type of smooth- ing is a better idea than the relative- frequency estimates that are often used. The best smoothing techniques yield con- sistent gains of approximately 1% (abso- lute) according to the BLEU metric. 1 Introduction Smoothing is an important technique in statistical NLP, used to deal with perennial data sparseness and empirical distributions that overfit the training corpus. Surprisingly, however, it is rarely men- tioned in statistical Machine Translation. In par- ticular, state-of-the-art phrase-based SMT relies on a phrasetable—a large set of ngram pairs over the source and target languages, along with their translation probabilities. This table, which may contain tens of millions of entries, and phrases of up to ten words or more, is an excellent candidate for smoothing. Yet very few publications describe phrasetable smoothing techniques in detail. In this paper, we provide the first system- atic study of smoothing methods for phrase-based SMT. Although we introduce a few new ideas, most methods described here were devised by oth- ers; the main purpose of this paper is not to in- vent new methods, but to compare methods. In experiments over many language pairs, we show that smoothing yields small but consistent gains in translation performance. We feel that this paper only scratches the surface: many other combina- tions of phrasetable smoothing techniques remain to be tested. We define a phrasetable as a set of source phrases (ngrams) ˜s and their translations ˜t, along with associated translation probabilities p(˜s|˜t) and p(˜t|˜s). These conditional distributions are derived from the joint frequencies c(˜s, ˜t) of source/target phrase pairs observed in a word-aligned parallel corpus. Traditionally, maximum-likelihood estimation from relative frequencies is used to obtain con- ditional probabilities (Koehn et al., 2003), eg, p(˜s|˜t) = c(˜s, ˜t)/ � ˜s c(˜s, ˜t) (since the estimation problems for p(˜s|˜t) and p(˜t|˜s) are symmetrical, we will usually refer only to p(˜s|˜t) for brevity). The most obvious example of the overfitting this causes can be seen in phrase pairs whose con- stituent phrases occur only once in the corpus. These are assigned conditional probabilities of 1, higher than the estimated probabilities of pairs for which much more evidence exists, in the typical case where the latter have constituents that co- occur occasionally with other phrases. During de- coding, overlapping phrase pairs are in direct com- petition, so estimation biases such as this one in favour of infrequent pairs have the potential to sig- nificantly degrade translation quality. An excellent discussion of smoothing tech- niques developed for ngram language models (LMs) may be found in (Chen and Goodman, 1998; Goodman, 2001). Phrasetable …

Content truncated to 500 words