Discriminative Word Alignment with Conditional Random Fields

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 65–72, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Discriminative Word Alignment with Conditional Random Fields Phil Blunsom and Trevor Cohn Department of Software Engineering and Computer Science University of Melbourne {pcbl,tacohn}@csse.unimelb.edu.au Abstract In this paper we present a novel approach for inducing word alignments from sen- tence aligned data. We use a Condi- tional Random Field (CRF), a discrimina- tive model, which is estimated on a small supervised training set. The CRF is condi- tioned on both the source and target texts, and thus allows for the use of arbitrary and overlapping features over these data. Moreover, the CRF has efficient training and decoding processes which both find globally optimal solutions. We apply this alignment model to both French-English and Romanian-English language pairs. We show how a large number of highly predictive features can be easily incorporated into the CRF, and demonstrate that even with only a few hun- dred word-aligned training sentences, our model improves over the current state-of- the-art with alignment error rates of 5.29 and 25.8 for the two tasks respectively. 1 Introduction Modern phrase based statistical machine transla- tion (SMT) systems usually break the translation task into two phases. The first phase induces word alignments over a sentence-aligned bilingual cor- pus, and the second phase uses statistics over these predicted word alignments to decode (translate) novel sentences. This paper deals with the first of these tasks: word alignment. Most current SMT systems (Och and Ney, 2004; Koehn et al., 2003) use a generative model for word alignment such as the freely available GIZA++ (Och and Ney, 2003), an implementa- tion of the IBM alignment models (Brown et al., 1993). These models treat word alignment as a hidden process, and maximise the probability of the observed (e, f) sentence pairs1 using the ex- pectation maximisation (EM) algorithm. After the maximisation process is complete, the word align- ments are set to maximum posterior predictions of the model. While GIZA++ gives good results when trained on large sentence aligned corpora, its generative models have a number of limitations. Firstly, they impose strong independence assumptions be- tween features, making it very difficult to incor- porate non-independent features over the sentence pairs. For instance, as well as detecting that a source word is aligned to a given target word, we would also like to encode syntactic and lexi- cal features of the word pair, such as their parts- of-speech, affixes, lemmas, etc. Features such as these would allow for more effective use of sparse data and result in a model which is more robust in the presence of unseen words. Adding these non-independent features to a generative model requires that the features’ inter-dependence be modelled explicitly, which often complicates the model (eg. Toutanova et al. (2002)). Secondly, the later IBM models, such as Model 4, have to re- sort to heuristic search techniques to approximate forward-backward and Viterbi inference, which sacrifice optimality for …

Content truncated to 500 words