Online Large-Margin Training of Syntactic and Structural Translation Features

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 224–233, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Online Large-Margin Training of Syntactic and Structural Translation Features David Chiang Information Sciences Institute University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292, USA chiang@isi.edu Yuval Marton and Philip Resnik Department of Linguistics and the UMIACS Laboratory for Computational Linguistics and Information Processing University of Maryland College Park, MD 20742, USA {ymarton,resnik}@umiacs.umd.edu Abstract Minimum-error-rate training (MERT) is a bot- tleneck for current development in statistical machine translation because it is limited in the number of weights it can reliably opti- mize. Building on the work of Watanabe et al., we explore the use of the MIRA algorithm of Crammer et al. as an alternative to MERT. We first show that by parallel processing and exploiting more of the parse forest, we can obtain results using MIRA that match or sur- pass MERT in terms of both translation qual- ity and computational cost. We then test the method on two classes of features that address deficiencies in the Hiero hierarchical phrase- based model: first, we simultaneously train a large number of Marton and Resnik’s soft syn- tactic constraints, and, second, we introduce a novel structural distortion model. In both cases we obtain significant improvements in translation performance. Optimizing them in combination, for a total of 56 feature weights, we improve performance by 2.6 B on a subset of the NIST 2006 Arabic-English eval- uation data. 1 Introduction Since its introduction by Och (2003), minimum er- ror rate training (MERT) has been widely adopted for training statistical machine translation (MT) sys- tems. However, MERT is limited in the number of feature weights that it can optimize reliably, with folk estimates of the limit ranging from 15 to 30 fea- tures. One recent example of this limitation is a series of experiments by Marton and Resnik (2008), in which they added syntactic features to Hiero (Chi- ang, 2005; Chiang, 2007), which ordinarily uses no linguistically motivated syntactic information. Each of their new features rewards or punishes a deriva- tion depending on how similar or dissimilar it is to a syntactic parse of the input sentence. They found that in order to obtain the greatest improve- ment, these features had to be specialized for par- ticular syntactic categories and weighted indepen- dently. Not being able to optimize them all at once using MERT, they resorted to running MERT many times in order to test different combinations of fea- tures. But it would have been preferable to use a training method that can optimize the features all at once. There has been much work on improving MERT’s performance (Duh and Kirchoff, 2008; Smith and Eisner, 2006; Cer et al., 2008), or on replacing MERT wholesale (Turian et al., 2007; Blunsom et al., 2008). This paper continues a line of research on online discriminative training (Tillmann and Zhang, 2006; Liang et al., 2006; Arun and Koehn, 2007), extending that …

Content truncated to 500 words