Maximum Entropy Translation Model in Dependency-Based MT Framework

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 201–206, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics Maximum Entropy Translation Model in Dependency-Based MT Framework David Mareˇcek, Martin Popel, Zdenˇek ˇZabokrtsk´y Charles University in Prague, Institute of Formal and Applied Linguistics Malostransk´e n´am. 25, Praha 1, CZ-118 00, Czech Republic {marecek,popel,zabokrtsky}@ufal.mff.cuni.cz Abstract Maximum Entropy Principle has been used successfully in various NLP tasks. In this paper we propose a forward transla- tion model consisting of a set of maxi- mum entropy classifiers: a separate clas- sifier is trained for each (sufficiently fre- quent) source-side lemma. In this way the estimates of translation probabilities can be sensitive to a large number of fea- tures derived from the source sentence (in- cluding non-local features, features mak- ing use of sentence syntactic structure, etc.). When integrated into English-to- Czech dependency-based translation sce- nario implemented in the TectoMT frame- work, the new translation model signif- icantly outperforms the baseline model (MLE) in terms of BLEU. The perfor- mance is further boosted in a configuration inspired by Hidden Tree Markov Mod- els which combines the maximum entropy translation model with the target-language dependency tree model. 1 Introduction The principle of maximum entropy states that, given known constraints, the probability distri- bution which best represents the current state of knowledge is the one with the largest entropy. Maximum entropy models based on this princi- ple have been widely used in Natural Language Processing, e.g. for tagging (Ratnaparkhi, 1996), parsing (Charniak, 2000), and named entity recog- nition (Bender et al., 2003). Maximum entropy models have the following form p(y|x) = 1 Z(x)exp � i λifi(x, y) where fi is a feature function, λi is its weight, and Z(x) is the normalizing factor Z(x) = � y exp � i λifi(x, y) In statistical machine translation (SMT), trans- lation model (TM) p(t|s) is the probability that the string t from the target language is the translation of the string s from the source language. Typical approach in SMT is to use backward translation model p(s|t) according to Bayes’ rule and noisy- channel model. However, in this paper we deal only with the forward (direct) model.1 The idea of using maximum entropy for con- structing forward translation models is not new. It naturally allows to make use of various features potentially important for correct choice of target- language expressions. Let us adopt a motivat- ing example of such a feature from (Berger et al., 1996) (which contains the first usage of maxent translation model we are aware of): “If house ap- pears within the next three words (e.g., the phrases in the house and in the red house), then dans might be a more likely [French] translation [of in].” Incorporating non-local features extracted from the source sentence into the standard noisy- channel model in which only the backward trans- lation model is available, is not possible. This drawback of the noisy-channel approach is typi- cally compensated by using large target-language n-gram models, …

Content truncated to 500 words