A Discriminative Matching Approach to Word Alignment

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 73–80, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics A Discriminative Matching Approach to Word Alignment Ben Taskar Simon Lacoste-Julien Dan Klein Computer Science Division, EECS Department University of California, Berkeley Berkeley, CA 94720 Abstract We present a discriminative, large- margin approach to feature-based matching for word alignment. In this framework, pairs of word tokens re- ceive a matching score, which is based on features of that pair, including mea- sures of association between the words, distortion between their positions, sim- ilarity of the orthographic form, and so on. Even with only 100 labeled train- ing examples and simple features which incorporate counts from a large unla- beled corpus, we achieve AER perfor- mance close to IBM Model 4, in much less time. Including Model 4 predic- tions as features, we achieve a relative AER reduction of 22% in over inter- sected Model 4 alignments. 1 Introduction The standard approach to word alignment from sentence-aligned bitexts has been to construct models which generate sentences of one lan- guage from the other, then fitting those genera- tive models with EM (Brown et al., 1990; Och and Ney, 2003). This approach has two primary advantages and two primary drawbacks. In its favor, generative models of alignment are well- suited for use in a noisy-channel translation sys- tem. In addition, they can be trained in an un- supervised fashion, though in practice they do require labeled validation alignments for tuning model hyper-parameters, such as null counts or smoothing amounts, which are crucial to pro- ducing alignments of good quality. A primary drawback of the generative approach to align- ment is that, as in all generative models, explic- itly incorporating arbitrary features of the in- put is difficult. For example, when considering whether to align two words in the IBM models (Brown et al., 1990), one cannot easily include information about such features as orthographic similarity (for detecting cognates), presence of the pair in various dictionaries, similarity of the frequency of the two words, choices made by other alignment systems on this sentence pair, and so on. While clever models can implicitly capture some of these information sources, it takes considerable work, and can make the re- sulting models quite complex. A second draw- back of generative translation models is that, since they are learned with EM, they require extensive processing of large amounts of data to achieve good performance. While tools like GIZA++ (Och and Ney, 2003) do make it eas- ier to build on the long history of the generative IBM approach, they also underscore how com- plex high-performance generative models can, and have, become. In this paper, we present a discriminative ap- proach to word alignment. Word alignment is cast as a maximum weighted matching problem (Cormen et al., 1990) in which each pair of words (ej, fk) in a sentence pair (e, f) is associated with a score sjk(e, f) reflecting the desirability …

Content truncated to 500 words