Lattice-based Minimum Error Rate Training for Statistical Machine Translation

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 725–734, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Lattice-based Minimum Error Rate Training for Statistical Machine Translation Wolfgang Macherey Franz Josef Och Ignacio Thayer Jakob Uszkoreit Google Inc. 1600 Amphitheatre Parkway Mountain View, CA 94043, USA {wmach,och,thayer,uszkoreit}@google.com Abstract Minimum Error Rate Training (MERT) is an effective means to estimate the feature func- tion weights of a linear model such that an automated evaluation criterion for measuring system performance can directly be optimized in training. To accomplish this, the training procedure determines for each feature func- tion its exact error surface on a given set of candidate translations. The feature function weights are then adjusted by traversing the error surface combined over all sentences and picking those values for which the resulting error count reaches a minimum. Typically, candidates in MERT are represented as N- best lists which contain the N most probable translation hypotheses produced by a decoder. In this paper, we present a novel algorithm that allows for efficiently constructing and repre- senting the exact error surface of all trans- lations that are encoded in a phrase lattice. Compared to N-best MERT, the number of candidate translations thus taken into account increases by several orders of magnitudes. The proposed method is used to train the feature function weights of a phrase-based statistical machine translation system. Experi- ments conducted on the NIST 2008 translation tasks show significant runtime improvements and moderate BLEU score gains over N-best MERT. 1 Introduction Many statistical methods in natural language pro- cessing aim at minimizing the probability of sen- tence errors. In practice, however, system quality is often measured based on error metrics that assign non-uniform costs to classification errors and thus go far beyond counting the number of wrong de- cisions. Examples are the mean average precision for ranked retrieval, the F-measure for parsing, and the BLEU score for statistical machine transla- tion (SMT). A class of training criteria that provides a tighter connection between the decision rule and the final error metric is known as Minimum Error Rate Training (MERT) and has been suggested for SMT in (Och, 2003). MERT aims at estimating the model parameters such that the decision under the zero-one loss func- tion maximizes some end-to-end performance mea- sure on a development corpus. In combination with log-linear models, the training procedure allows for a direct optimization of the unsmoothed error count. The criterion can be derived from Bayes’ decision rule as follows: Let f � f1, ..., fJ denote a source sentence (’French’) which is to be translated into a target sentence (’English’) e � e1, ..., eI. Under the zero-one loss function, the translation which maximizes the a posteriori probability is chosen: ˆe � arg max e Prpe|f q ( (1) Since the true posterior distribution is unknown, Prpe|f q is modeled via a log-linear translation model which combines some feature functions hm pe, f q with feature function weights λm, m � 1, ..., …

Content truncated to 500 words