Improving Statistical Machine Translation with Word Class Models

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1377–1381, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Improving Statistical Machine Translation with Word Class Models Joern Wuebker, Stephan Peitz, Felix Rietig and Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University Aachen, Germany <surname>@cs.rwth-aachen.de Abstract Automatically clustering words from a mono- lingual or bilingual training corpus into classes is a widely used technique in statisti- cal natural language processing. We present a very simple and easy to implement method for using these word classes to improve trans- lation quality. It can be applied across differ- ent machine translation paradigms and with arbitrary types of models. We show its ef- ficacy on a small German→English and a larger French→German translation task with both standard phrase-based and hierarchical phrase-based translation systems for a com- mon set of models. Our results show that with word class models, the baseline can be im- proved by up to 1.4% BLEU and 1.0% TER on the French→German task and 0.3% BLEU and 1.1% TER on the German→English task. 1 Introduction Data sparsity is one of the major problems for statis- tical learning methods in natural language process- ing (NLP) today. Even with the huge training data sets available in some tasks, for many phenomena that need to be modeled only few training instances can be observed. This is partly due to the large vo- cabularies of natural languages. One possiblity to reduce the sparsity for model estimation is to re- duce the vocabulary size. By clustering the vocab- ulary into a fixed number of word classes, it is pos- sible to train models that are less prone to sparsity issues. This work investigates the performance of standard models used in statistical machine transla- tion when they are trained on automatically learned word classes rather than the actual word identities. In the popular tooklit GIZA++ (Och and Ney, 2003), word classes are an essential ingredient to model alignment probabilities with the HMM or IBM translation models. It contains the mkcls tool (Och, 1999), which can automatically cluster the vo- cabulary into classes. Using this tool, we propose to re-parameterize the standard models used in statistical machine transla- tion (SMT), which are usually conditioned on word identities rather than word classes. The idea is that this should lead to a smoother distribution, which is more reliable due to less sparsity. Here, we fo- cus on the phrase-based and lexical channel models in both directions, simple count models identifying frequency thresholds, lexicalized reordering models and an n-gram language model. Although our re- sults show that it is not a good idea to replace the original models, we argue that adding them to the log-linear feature combination can improve transla- tion quality. They can easily be computed for dif- ferent translation paradigms and arbitrary models. Training and decoding is possible without or with only little change to the code base. Our experiments are conducted on a medium- sized French→German …

Content truncated to 500 words