Considerations in maximum mutual information and minimum classification error training for statistical machine translation

EAMT 2005 Conference Proceedings 271 Considerations in Maximum Mutual Information and Minimum Classification Error training for Statistical Machine Translation Ashish Venugopal* and Stephan Vogel** Language Technologies Institute Carnegie Mellon University {ashishv;stephan.vogel}@cs.cmu.edu Abstract. Discriminative training methods are used in statistical machine translation to ef- fectively introduce and combine additional knowledge sources within the translation proc- ess. Although these methods are described in the accompanying literature and comparative studies are available for speech recognition, additional considerations are introduced when applying discriminative training to statistical machine translation. In this paper we pay spe- cial attention to the comparison and formalization of discriminative training criteria and their respective optimization methods with the goal of improving translation performance measured by the corpus level BLEU metric for a Viterbi beam based decoder. We frame this work within the current trends in discriminative training and present reproducible re- sults that highlight the potential as well as shortcomings of N-Best list based discriminative training. 1. Introduction Statistical machine translation, like other natu- ral language process tasks, has developed a set of unique evaluation metrics that go beyond simply evaluating the number of sentence errors that a system makes on a test set. While debates continue regarding the relative value of each competing metric, the BLEU [Papeneni, 2001] and NIST [Doddington, 2002] scores (which consider system performance at the corpus le- vel) have shown their effectiveness in driving the development of statistical machine transla- tion systems. These metrics have highlighted the need for more expressive models of translation and a framework to introduce additional knowl- edge sources within the translation process. The direction translation [Och, 2002] approach [Brown et al, 1993] delivers this framework, and pro- vides a necessary formalism to the process of combining and optimizing additional knowledge sources. Discriminative training considers com- peting candidate translations from an N-Best list is used to find appropriate scaling factors for each additional knowledge source. The goal is to find scaling factors that improve the metric performance of the candidate translation chosen by the decoder [Vogel, 2003]. In this paper, we consider these scaling factors within the decod- ing process rather than as a post processing re- ranking step, thereby creating additional con- siderations regarding the stability of the scaling factors. The choice of evaluation metric, the na- ture of the additional knowledge sources within the decoding process and the implementation decisions taken in each component, determine the effectiveness of each discriminative method. This paper will focus on comparing the for- malism and practical considerations involved with deploying Maximum Mutual Information [Bas- sat,1982] and Minimum Classification Error [Huang, Katagiri, 1992] training within a statis- tical machine translation context. We begin by framing the discriminative training task for sta- tistical machine translation and survey direc- tions of active research in the field. We discuss the impact that the corpus level BLEU score has on the discriminative training criteria and the implementation requirements for optimization methods that accommodate for such metrics. We describe the process of generating N-Best lists from a Viterbi decoding …

Content truncated to 500 words