Unsupervised Word Alignment with Arbitrary Features

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 409–419, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Unsupervised Word Alignment with Arbitrary Features Chris Dyer Jonathan Clark Alon Lavie Language Technologies InstituteNoah A. Smith Carnegie Mellon University Pittsburgh, PA 15213, USA {cdyer,jhclark,alavie,nasmith}@cs.cmu.edu Abstract We introduce a discriminatively trained, glob- ally normalized, log-linear variant of the lex- ical translation models proposed by Brown et al. (1993). In our model, arbitrary, non- independent features may be freely incorpo- rated, thereby overcoming the inherent limita- tion of generative models, which require that features be sensitive to the conditional inde- pendencies of the generative process. How- ever, unlike previous work on discriminative modeling of word alignment (which also per- mits the use of arbitrary features), the param- eters in our models are learned from unanno- tated parallel sentences, rather than from su- pervised word alignments. Using a variety of intrinsic and extrinsic measures, including translation performance, we show our model yields better alignments than generative base- lines in a number of language pairs. 1 Introduction Word alignment is an important subtask in statis- tical machine translation which is typically solved in one of two ways. The more common approach uses a generative translation model that relates bilin- gual string pairs using a latent alignment variable to designate which source words (or phrases) generate which target words. The parameters in these models can be learned straightforwardly from parallel sen- tences using EM, and standard inference techniques can recover most probable alignments (Brown et al., 1993). This approach is attractive because it only requires parallel training data. An alternative to the generative approach uses a discriminatively trained alignment model to predict word alignments in the parallel corpus. Discriminative models are attractive because they can incorporate arbitrary, overlapping features, meaning that errors observed in the predic- tions made by the model can be addressed by engi- neering new and better features. Unfortunately, both approaches are problematic, but in different ways. In the case of discriminative alignment mod- els, manual alignment data is required for train- ing, which is problematic for at least three reasons. Manual alignments are notoriously difficult to cre- ate and are available only for a handful of language pairs. Second, manual alignments impose a commit- ment to a particular preprocessing regime; this can be problematic since the optimal segmentation for translation often depends on characteristics of the test set or size of the available training data (Habash and Sadat, 2006) or may be constrained by require- ments of other processing components, such parsers. Third, the “correct” alignment annotation for differ- ent tasks may vary: for example, relatively denser or sparser alignments may be optimal for different ap- proaches to (downstream) translation model induc- tion (Lopez, 2008; Fraser, 2007). Generative models have a different limitation: the joint probability of a particular setting of the ran- dom variables must factorize according to steps in a process that successively “generates” the values of the variables. At each step, the probability …

Content truncated to 500 words