Statistical Machine Translation with Local Language Models

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 869–879, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Statistical Machine Translation with Local Language Models Christof Monz Informatics Institute, University of Amsterdam P.O. Box 94323, 1090 GH Amsterdam, The Netherlands c.monz@uva.nl Abstract Part-of-speech language modeling is com- monly used as a component in statistical ma- chine translation systems, but there is mixed evidence that its usage leads to significant im- provements. We argue that its limited effec- tiveness is due to the lack of lexicalization. We introduce a new approach that builds a separate local language model for each word and part-of-speech pair. The resulting mod- els lead to more context-sensitive probabil- ity distributions and we also exploit the fact that different local models are used to esti- mate the language model probability of each word during decoding. Our approach is evalu- ated for Arabic- and Chinese-to-English trans- lation. We show that it leads to statistically significant improvements for multiple test sets and also across different genres, when com- pared against a competitive baseline and a sys- tem using a part-of-speech model. 1 Introduction Language models are an important component of current statistical machine translation systems. They affect the selection of phrase translation candidates and reordering choices by estimating the probability that an application of a phrase translation is a flu- ent continuation of the current translation hypoth- esis. The size and domain of the language model can have a significant impact on translation quality. Brants et al. (2007) have shown that each doubling of the training data from the news domain (used to build the language model), leads to improvements of approximately 0.5 BLEU points. On the other hand, each doubling using general web data leads to im- provements of approximately 0.15 BLEU points. While large n-gram language models do lead to improved translation quality, they still lack any generalization beyond the surface forms (Schwenk, 2007). Consider example (1), which is a short sen- tence fragment from the MT09 Arabic-English test set, with the corresponding machine translation out- put (1.b), from a phrase-based statistical machine translation system, and reference translation (1.c). (1) a. ���� �� �������� ���������� ������������ ������� �� ... ... �� ��������� b. ... the background of press statements of controversial and accused him ... c. ... the background of controversial press statements and accused him ... Clearly, the adjective “controversial” should pre- cede the nouns “press statement”, but since the AFP and Xinhua portions of the Gigaword corpus, used to build the language model for the translation sys- tem, do not contain this surface n-gram, translations with obviously ungrammatical constructions such as (1.b) can result. For unseen n-grams, one would like to model adjectives as being likely to precede nouns in English, for example. A straightforward approach to address this is to exploit the part-of-speech (POS) tags of the tar- get words during translation (Kirchhoff and Yang, 2005). Though models exploiting POS information are not expressive enough to model …

Content truncated to 500 words