Factored Translation Models

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 868–876, Prague, June 2007. c⃝2007 Association for Computational Linguistics Factored Translation Models Philipp Koehn and Hieu Hoang pkoehn@inf.ed.ac.uk, H.Hoang@sms.ed.ac.uk School of Informatics University of Edinburgh 2 Buccleuch Place, Edinburgh EH8 9LW Scotland, United Kingdom Abstract We present an extension of phrase-based statistical machine translation models that enables the straight-forward integration of additional annotation at the word-level — may it be linguistic markup or automati- cally generated word classes. In a num- ber of experiments we show that factored translation models lead to better transla- tion performance, both in terms of auto- matic scores, as well as more grammatical coherence. 1 Introduction The current state-of-the-art approach to statistical machine translation, so-called phrase-based models, is limited to the mapping of small text chunks with- out any explicit use of linguistic information, may it be morphological, syntactic, or semantic. Such additional information has been demonstrated to be valuable by integrating it in pre-processing or post- processing steps. However, a tighter integration of linguistic infor- mation into the translation model is desirable for two reasons: • Translation models that operate on more gen- eral representations, such as lemmas instead of surface forms of words, can draw on richer statistics and overcome the data sparseness problems caused by limited training data. • Many aspects of translation can be best ex- plained on a morphological, syntactic, or se- mantic level. Having such information avail- able to the translation model allows the direct modeling of these aspects. For instance: re- ordering at the sentence level is mostly driven word word part-of-speech Output Input morphology part-of-speech morphology word class lemma word class lemma ... ... Figure 1: Factored representations of input and out- put words incorporate additional annotation into the statistical translation model. by general syntactic principles, local agreement constraints show up in morphology, etc. Therefore, we extended the phrase-based ap- proach to statistical translation to tightly integrate additional information. The new approach allows additional annotation at the word level. A word in our framework is not only a token, but a vector of factors that represent different levels of annotation (see Figure 1). We report on experiments with factors such as surface form, lemma, part-of-speech, morphologi- cal features such as gender, count and case, auto- matic word classes, true case forms of words, shal- low syntactic tags, as well as dedicated factors to en- sure agreement between syntactically related items. This paper describes the motivation, the modeling aspects and the computationally efficient decoding methods of factored translation models. We present briefly results for a number of language pairs. How- ever, the focus of this paper is the description of the approach. Detailed experimental results will be de- scribed in forthcoming papers. 868 2 Related Work Many attempts have been made to add richer in- formation to statistical machine translation models. Most of these focus on the pre-processing of the in- put to the statistical system, or the post-processing …

Content truncated to 500 words