Language Model Adaptation for Statistical Machine Translation via Structured Query Models

Language Model Adaptation for Statistical Machine Translation with Structured Query Models Bing Zhao Matthias Eck Stephan Vogel Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA, 15213, USA {bzhao, matteck, vogel+}@cs.cmu.edu Abstract We explore unsupervised language model adaptation techniques for Statistical Machine Translation. The hypotheses from the machine translation output are converted into queries at different levels of representation power and used to extract similar sentences from very large monolingual text collection. Specific language models are then build from the retrieved data and interpolated with a general background model. Experiments show significant improvements when translating with these adapted language models. 1 Introduction Language models (LM) are applied in many natural language processing applications, such as speech recognition and machine translation, to encapsulate syntactic, semantic and pragmatic information. For systems which learn from given data we frequently observe a severe drop in performance when moving to a new genre or new domain. In speech recognition a number of adaptation techniques have been developed to cope with this situation. In statistical machine translation we have a similar situation, i.e. estimate the model parameter from some data, and use the system to translate sentences which may not be well covered by the training data. Therefore, the potential of adaptation techniques needs to be explored for machine translation applications. Statistical machine translation is based on the noisy channel model, where the translation hypothesis is searched over the space defined by a translation model and a target language (Brown et al, 1993). Statistical machine translation can be formulated as follows: () argmax(|) argmax(|) * Pt Pst Pts t t t ⋅ = = where t is the target sentence, and s is the source sentence. P(t) is the target language model and P(s|t) is the translation model. The argmax operation is the search, which is done by the decoder. In the current study we modify the target language model P(t), to represent the test data better, and thereby improve the translation quality. (Janiszek, et al. 2001) list the following approaches • to language model adaptation: Linear interpolation of a general and a domain specific model (Seymore, Rosenfeld, 1997). • Back off of domain specific probabilities with those of a specific model (Besling, Meier, 1995). • Retrieval of documents pertinent to the new domain and training a language model on-line with those data (Iyer, Ostendorf, 1999, Mahajan et. al. 1999). • Maximum entropy, minimum discrimination adaptation (Chen, et. al., 1998). • Adaptation by linear transformation of vectors of bigram counts in a reduced space (DeMori, Federico, 1999). • Smoothing and adaptation in a dual space via latent semantic analysis, modeling long-term semantic dependencies, and trigger combinations. (J. Bellegarda, 2000). Our approach can be characterized as unsupervised data augmentation by retrieval of relevant documents from large monolingual corpora, and interpolation of the specific language model, build from the retrieved data, with a background language model. To be more specific, the following steps are carried out to do the language model adaptation. First, a baseline statistical machine translation system, …

Content truncated to 500 words