Scalable Purely-Discriminative Training for Word and Tree Transducers

Scalable Purely-Discriminative Training for Word and Tree Transducers Benjamin Wellington, Joseph Turian, Chris Pike, and I. Dan Melamed ∗ Computer Science Department New York University {lastname}@cs.nyu.edu Abstract Discriminative training methods have re- cently led to significant advances in the state of the art of machine translation (MT). Another promising trend is the in- corporation of syntactic information into MT systems. Combining these trends is difficult for reasons of system complex- ity and computational complexity. The present study makes progress towards a syntax-aware MT system whose ev- ery component is trained discriminatively. Our main innovation is an approach to discriminative learning that is computa- tionally efficient enough for large statis- tical MT systems, yet whose accuracy on translation sub-tasks is near the state of the art. Our source code is down- loadable from http://nlp.cs.nyu. edu/GenPar/. 1 Introduction Discriminative training methods have recently led to significant advances in the state of the art of ma- chine translation (MT). Another promising trend is the incorporation of syntactic information into MT systems. Combining these trends is difficult for rea- sons of system complexity and computational com- plexity. The present study makes progress towards a syntax-aware MT system whose every component is trained discriminatively. Our main innovation is an ∗ Many thanks to Leon Bottou for help with LaSVM. This research was supported by NSF grants #0238406 and #0415933. approach to discriminative learning that is computa- tionally efficient enough for large statistical MT sys- tems, yet whose accuracy on translation sub-tasks is near the state of the art. The core of our proposal is described in Section 2, and evaluated in Section 3. Several studies have explored discriminative training for machine translation, following the method of Och and Ney (2002). Several authors have even applied this method to syntax-aware sys- tems (Chiang, 2005; Quirk et al., 2005). However, Och’s method applies discriminative training only to a handful of meta-parameters that are used to combine information from a variety of sub-models. The sub-models may or may not be optimized for the same objective as the meta-parameters. On a task like machine translation of common languages, where training data is abundant, a more elegant and more accurate system might be obtained by training all parts of the system discriminatively (Ng and Jor- dan, 2002). Foster (2000) built such a system, but could not fully exploit the benefits of discriminative training for lack of a suitable regularization scheme. 2 Training Method 2.1 The Training Set Predicting a target tree given a source tree is equiv- alent to predicting a synchronous tree (“bitree”) that is consistent with the source tree. Our method for training tree transducers was to train an inference engine to predict bitrees. Our method for predicting bitrees is to make a sequence of multiclass classifi- cation decisions called inferences. Each inference predicts an unstructured part of the eventual bitree. Thus, to train a model for predicting bitrees, it is sufficient to train it to predict correct inferences. 251 Proceedings of the 7th Conference of the Association …

Content truncated to 500 words