CCG Supertags in Factored Statistical Machine Translation

Proceedings of the Second Workshop on Statistical Machine Translation, pages 9–16, Prague, June 2007. c⃝2007 Association for Computational Linguistics CCG Supertags in Factored Statistical Machine Translation Alexandra Birch Miles Osborne Philipp Koehn a.c.birch-mayne@sms.ed.ac.uk miles@inf.ed.ac.uk pkoehn@inf.ed.ac.uk School of Informatics University of Edinburgh 2 Buccleuch Place Edinburgh, EH8 9LW, UK Abstract Combinatorial Categorial Grammar (CCG) supertags present phrase-based machine translation with an opportunity to access rich syntactic information at a word level. The challenge is incorporating this informa- tion into the translation process. Factored translation models allow the inclusion of su- pertags as a factor in the source or target lan- guage. We show that this results in an im- provement in the quality of translation and that the value of syntactic supertags in flat structured phrase-based models is largely due to better local reorderings. 1 Introduction In large-scale machine translation evaluations, phrase-based models generally outperform syntax- based models1. Phrase-based models are effective because they capture the lexical dependencies be- tween languages. However, these models, which are equivalent to finite-state machines (Kumar and Byrne, 2003), are unable to model long range word order differences. Phrase-based models also lack the ability to incorporate the generalisations implicit in syntactic knowledge and they do not respect linguis- tic phrase boundaries. This makes it difficult to im- prove reordering in phrase-based models. Syntax-based models can overcome some of the problems associated with phrase-based models be- cause they are able to capture the long range struc- tural mappings that occur in translation. Recently 1www.nist.gov/speech/tests/mt/mt06eval official results.html there have been a few syntax-based models that show performance comparable to the phrase-based models (Chiang, 2005; Marcu et al., 2006). How- ever, reliably learning powerful rules from parallel data is very difficult and prone to problems with sparsity and noise in the data. These models also suffer from a large search space when decoding with an integrated language model, which can lead to search errors (Chiang, 2005). In this paper we investigate the idea of incorporat- ing syntax into phrase-based models, thereby lever- aging the strengths of both the phrase-based models and syntactic structures. This is done using CCG supertags, which provide a rich source of syntactic information. CCG contains most of the structure of the grammar in the lexicon, which makes it possi- ble to introduce CCG supertags as a factor in a fac- tored translation model (Koehn et al., 2006). Fac- tored models allow words to be vectors of features: one factor could be the surface form and other fac- tors could contain linguistic information. Factored models allow for the easy inclusion of supertags in different ways. The first approach is to generate CCG supertags as a factor in the target and then apply an n-gram model over them, increasing the probability of more frequently seen sequences of supertags. This is a simple way of including syn- tactic information in a phrase-based model, and has also been suggested by Hassan et al. (2007). For both Arabic-English (Hassan et al., 2007) and our experiments in Dutch-English, n-gram models over …

Content truncated to 500 words