Stochastic Lexicalized Inversion Transduction Grammar for Alignment

Proceedings of the 43rd Annual Meeting of the ACL, pages 475–482, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Stochastic Lexicalized Inversion Transduction Grammar for Alignment Hao Zhang and Daniel Gildea Computer Science Department University of Rochester Rochester, NY 14627 Abstract We present a version of Inversion Trans- duction Grammar where rule probabili- ties are lexicalized throughout the syn- chronous parse tree, along with pruning techniques for efficient training. Align- ment results improve over unlexicalized ITG on short sentences for which full EM is feasible, but pruning seems to have a negative impact on longer sentences. 1 Introduction The Inversion Transduction Grammar (ITG) of Wu (1997) is a syntactically motivated algorithm for producing word-level alignments of pairs of transla- tionally equivalent sentences in two languages. The algorithm builds a synchronous parse tree for both sentences, and assumes that the trees have the same underlying structure but that the ordering of con- stituents may differ in the two languages. This probabilistic, syntax-based approach has in- spired much subsequent reasearch. Alshawi et al. (2000) use hierarchical finite-state transducers. In the tree-to-string model of Yamada and Knight (2001), a parse tree for one sentence of a transla- tion pair is projected onto the other string. Melamed (2003) presents algorithms for synchronous parsing with more complex grammars, discussing how to parse grammars with greater than binary branching and lexicalization of synchronous grammars. Despite being one of the earliest probabilistic syntax-based translation models, ITG remains state- of-the art. Zens and Ney (2003) found that the con- straints of ITG were a better match to the decod- ing task than the heuristics used in the IBM decoder of Berger et al. (1996). Zhang and Gildea (2004) found ITG to outperform the tree-to-string model for word-level alignment, as measured against human gold-standard alignments. One explanation for this result is that, while a tree representation is helpful for modeling translation, the trees assigned by the traditional monolingual parsers (and the treebanks on which they are trained) may not be optimal for translation of a specific language pair. ITG has the advantage of being entirely data-driven – the trees are derived from an expectation maximization pro- cedure given only the original strings as input. In this paper, we extend ITG to condition the grammar production probabilities on lexical infor- mation throughout the tree. This model is reminis- cent of lexicalization as used in modern statistical parsers, in that a unique head word is chosen for each constituent in the tree. It differs in that the head words are chosen through EM rather than de- terministic rules. This approach is designed to retain the purely data-driven character of ITG, while giving the model more information to work with. By condi- tioning on lexical information, we expect the model to be able capture the same systematic differences in languages’ grammars that motive the tree-to-string model, for example, SVO vs. SOV word order or prepositions vs. postpositions, but to be able to do so in a more fine-grained manner. The interaction …

Content truncated to 500 words