One Decade of Statistical Machine Translation: 1996-2005

One Decade of Statistical Machine Translation: 1996–2005 Hermann Ney Human Language Technology and Pattern Recognition Lehrstuhl f¨ur Informatik VI – Computer Science Department RWTH Aachen University D-52056 Aachen, Germany ney@informatik.rwth-aachen.de Abstract In the last decade, the statistical approach has found widespread use in machine translation both for written and spoken language and has had a major impact on the translation accuracy. This paper will cover the principles of statistical machine translation and summarize the progress made so far. 1 Introduction The goal of this paper is to cover the state of the art in statistical machine translation (MT). We will re-visit the underlying principles of the statistical approach to machine translation (and other tasks in natural language processing) and summarize the progress that has been made over the last decade. 2 Recasting MT as a Statistical Problem 2.1 The Baseline Approach to Statistical MT When translating a sentence from a source language into a target language, a human translator has to take into account the following considerations: • Lexical choice: Typically, a source word can have several translations in the target language, and the selection of the suitable translation depends on the context. • Position re-ordering: The word positions in source and target sentence are different. • Syntax and semantics: To generate the target sentence, syntactic and semantic constraints have to be taken into account. The corresponding operations are summarized in Fig. 1. The traditional non-statistical approach to MT is to manually design rules and knowledge sources for these operations. There are two open problems with this concept: 1. How can we get hold of all the rules and make sure that we know all the rules that the system needs? 2. How can we achieve a coherent and consistent interaction of all these rules when generating the target sentence? Both problems are elegantly addressed in the statistical approach. Instead of using hard rules, we make use of probability distributions that serve as probabilistic knowledge sources. In the baseline version of the statistical approach as introduced by IBM (Berger et al. 1994; Brown et al. 1993), the translation task can be expressed as follows. We are given a sentence F in the source language and we want to generate the corresponding target sentence E. For this purpose, we consider the associated posterior distribution p(F|E) of all possible pairs (F, E) and select the target sentence ˆE(F) with the highest posterior probability: F → ˆE(F) = arg max E {p(E|F)} = arg max E {p(E) · p(F|E)} which is referred to as Bayes decision rule. In the second equation, the posterior probability p(F|E) has been replaced by the joint probability p(E, F) which is written as the product of the so-called language model p(E) and the so-called translation model p(F|E). The translation model is still rather complex because it describes probability distributions over whole sentences F and E. To reduce its complexity, we make use of the concept of word alignments A that capture the correspondences between source and target words …

Content truncated to 500 words