Discriminative Modeling of Extraction Sets for Machine Translation

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1453–1463, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Discriminative Modeling of Extraction Sets for Machine Translation John DeNero and Dan Klein Computer Science Division University of California, Berkeley {denero,klein}@cs.berkeley.edu Abstract We present a discriminative model that di- rectly predicts which set of phrasal transla- tion rules should be extracted from a sen- tence pair. Our model scores extraction sets: nested collections of all the overlap- ping phrase pairs consistent with an under- lying word alignment. Extraction set mod- els provide two principle advantages over word-factored alignment models. First, we can incorporate features on phrase pairs, in addition to word links. Second, we can optimize for an extraction-based loss function that relates directly to the end task of generating translations. Our model gives improvements in alignment quality relative to state-of-the-art unsuper- vised and supervised baselines, as well as providing up to a 1.4 improvement in BLEU score in Chinese-to-English trans- lation experiments. 1 Introduction In the last decade, the field of statistical machine translation has shifted from generating sentences word by word to systems that recycle whole frag- ments of training examples, expressed as transla- tion rules. This general paradigm was first pur- sued using contiguous phrases (Och et al., 1999; Koehn et al., 2003), and has since been general- ized to a wide variety of hierarchical and syntactic formalisms. The training stage of statistical sys- tems focuses primarily on discovering translation rules in parallel corpora. Most systems discover translation rules via a two-stage pipeline: a parallel corpus is aligned at the word level, and then a second procedure ex- tracts fragment-level rules from word-aligned sen- tence pairs. This paper offers a model-based alter- native to phrasal rule extraction, which merges this two-stage pipeline into a single step. We present a discriminative model that directly predicts which set of phrasal translation rules should be extracted from a sentence pair. Our model predicts extrac- tion sets: combinatorial objects that include the set of all overlapping phrasal translation rules con- sistent with an underlying word-level alignment. This approach provides additional discriminative power relative to word aligners because extraction sets are scored based on the phrasal rules they con- tain in addition to word-to-word alignment links. Moreover, the structure of our model directly re- flects the purpose of alignment models in general, which is to discover translation rules. We address several challenges to training and applying an extraction set model. First, we would like to leverage existing word-level alignment re- sources. To do so, we define a deterministic map- ping from word alignments to extraction sets, in- spired by existing extraction procedures. In our mapping, possible alignment links have a precise interpretation that dictates what phrasal translation rules can be extracted from a sentence pair. This mapping allows us to train with existing annotated data sets and use the predictions from word-level aligners as features in our extraction set model. Second, our model solves a structured predic- tion …

Content truncated to 500 words