Mixture-Model Adaptation for SMT

Proceedings of the Second Workshop on Statistical Machine Translation, pages 128–135, Prague, June 2007. c⃝2007 Association for Computational Linguistics Mixture-Model Adaptation for SMT George Foster and Roland Kuhn National Research Council Canada first.last@nrc.gc.ca Abstract We describe a mixture-model approach to adapting a Statistical Machine Translation System for new domains, using weights that depend on text distances to mixture compo- nents. We investigate a number of variants on this approach, including cross-domain versus dynamic adaptation; linear versus loglinear mixtures; language and transla- tion model adaptation; different methods of assigning weights; and granularity of the source unit being adapted to. The best methods achieve gains of approximately one BLEU percentage point over a state-of-the art non-adapted baseline system. 1 Introduction Language varies significantly across different gen- res, topics, styles, etc. This affects empirical mod- els: a model trained on a corpus of car-repair manu- als, for instance, will not be well suited to an appli- cation in the field of tourism. Ideally, models should be trained on text that is representative of the area in which they will be used, but such text is not al- ways available. This is especially the case for bilin- gual applications, because parallel training corpora are relatively rare and tend to be drawn from spe- cific domains such as parliamentary proceedings. In this paper we address the problem of adapting a statistical machine translation system by adjust- ing its parameters based on some information about a test domain. We assume two basic settings. In cross-domain adaptation, a small sample of parallel in-domain text is available, and it is used to optimize for translating future texts drawn from the same do- main. In dynamic adaptation, no domain informa- tion is available ahead of time, and adaptation is based on the current source text under translation. Approaches developed for the two settings can be complementary: an in-domain development corpus can be used to make broad adjustments, which can then be fine tuned for individual source texts. Our method is based on the classical technique of mixture modeling (Hastie et al., 2001). This involves dividing the training corpus into different components, training a model on each part, then weighting each model appropriately for the current context. Mixture modeling is a simple framework that encompasses many different variants, as de- scribed below. It is naturally fairly low dimensional, because as the number of sub-models increases, the amount of text available to train each, and therefore its reliability, decreases. This makes it suitable for discriminative SMT training, which is still a chal- lenge for large parameter sets (Tillmann and Zhang, 2006; Liang et al., 2006). Techniques for assigning mixture weights depend on the setting. In cross-domain adaptation, knowl- edge of both source and target texts in the in-domain sample can be used to optimize weights directly. In dynamic adaptation, training poses a problem be- cause no reference text is available. Our solution is to construct a multi-domain development sample for learning parameter settings that are intended to generalize to new domains …

Content truncated to 500 words