Discriminative Word Alignment with a Function Word Reordering Model

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 534–544, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Discriminative Word Alignment with a Function Word Reordering Model Hendra Setiawan UMIACS University of Maryland hendra@umiacs.umd.edu Chris Dyer Language Technologies Institute Carnegie Mellon University cdyer@cs.cmu.edu Philip Resnik Linguistics and UMIACS University of Maryland resnik@umd.edu Abstract We address the modeling, parameter estima- tion and search challenges that arise from the introduction of reordering models that capture non-local reordering in alignment modeling. In particular, we introduce several reordering models that utilize (pairs of) function words as contexts for alignment reordering. To ad- dress the parameter estimation challenge, we propose to estimate these reordering models from a relatively small amount of manually- aligned corpora. To address the search chal- lenge, we devise an iterative local search al- gorithm that stochastically explores reorder- ing possibilities. By capturing non-local re- ordering phenomena, our proposed alignment model bears a closer resemblance to state- of-the-art translation model. Empirical re- sults show significant improvements in align- ment quality as well as in translation perfor- mance over baselines in a large-scale Chinese- English translation task. 1 Introduction In many Statistical Machine Translation (SMT) sys- tems, alignment represents an important piece of in- formation, from which translation rules are learnt. However, while translation models have evolved from word-based to syntax-based modeling, the de facto alignment model remains word-based (Brown et al., 1993; Vogel et al., 1996). This gap be- tween alignment modeling and translation modeling is clearly undesirable as it often generates tensions that would prevent the extraction of many useful translation rules (DeNero and Klein, 2007). Recent work, e.g. by Blunsom et al. (2009) and Haghihi et al. (2009) just to name a few, show that alignment models that bear closer resemblance to state-of-the- art translation model consistently yields not only a better alignment quality but also an improved trans- lation quality. In this paper, we follow this recent effort to nar- row the gap between alignment model and trans- lation model to improve translation quality. More concretely, we focus on the reordering component since we observe that the treatment of reordering re- mains significantly different when comparing align- ment versus translation: the reordering component in state-of-the-art translation models has focused on long-distance reordering, but its counterpart in alignment models has remained focused on local reordering, typically modeling distortion based en- tirely on positional information. This leaves most alignment decisions to association-based scores. Why is employing stronger reordering models more challenging in alignment than in translation? One answer can be attributed to the fact that align- ment points are unobserved in parallel text, thus so are their reorderings. As such, introducing stronger reordering often further exacerbates the computa- tional complexity to do inference over the model. Some recent alignment models appeal to external linguistic knowledge, mostly by using monolingual syntactic parses (Cherry and Lin, 2006; Pauls et al., 2010), which at the same time, provides an approx- imation of the bilingual syntactic divergences that …

Content truncated to 500 words