Latent-Variable Synchronous CFGs for Hierarchical Translation

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1953–1964, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Latent-Variable Synchronous CFGs for Hierarchical Translation Avneesh Saluja and Chris Dyer Carnegie Mellon University Pittsburgh, PA, 15213, USA {avneesh,cdyer}@cs.cmu.edu Shay B. Cohen University of Edinburgh Edinburgh EH8 9AB, UK scohen@inf.ed.ac.uk Abstract Data-driven refinement of non-terminal categories has been demonstrated to be a reliable technique for improving mono- lingual parsing with PCFGs. In this pa- per, we extend these techniques to learn latent refinements of single-category syn- chronous grammars, so as to improve translation performance. We compare two estimators for this latent-variable model: one based on EM and the other is a spec- tral algorithm based on the method of mo- ments. We evaluate their performance on a Chinese–English translation task. The re- sults indicate that we can achieve signifi- cant gains over the baseline with both ap- proaches, but in particular the moments- based estimator is both faster and performs better than EM. 1 Introduction Translation models based on synchronous context- free grammars (SCFGs) treat the translation prob- lem as a context-free parsing problem. A parser constructs trees over the input sentence by pars- ing with the source language projection of a syn- chronous CFG, and each derivation induces trans- lations in the target language (Chiang, 2007). However, in contrast to syntactic parsing, where linguistic intuitions can help elucidate the “right” tree structure for a grammatical sentence, no such intuitions are available for synchronous deriva- tions, and so learning the “right” grammars is a central challenge. Of course, learning synchronous grammars from parallel data is a widely studied problem (Wu, 1997; Blunsom et al., 2008; Levenberg et al., 2012, inter alia). However, there has been less exploration of learning rich non-terminal cat- egories, largely because previous efforts to learn such categories have been coupled with efforts to learn derivation structures—a computationally formidable challenge. One popular approach has been to derive categories from source and/or target monolingual grammars (Galley et al., 2004; Zoll- mann and Venugopal, 2006; Hanneman and Lavie, 2013). While often successful, accurate parsers are not available in many languages: a more ap- pealing approach is therefore to learn the category structure from the data itself. In this work, we take a different approach to previous work in synchronous grammar induc- tion by assuming that reasonable tree structures for a parallel corpus can be chosen heuristically, and then, fixing the trees (thereby enabling us to sidestep the worst of the computational issues), we learn non-terminal categories as latent variables to explain the distribution of these synchronous trees. This technique has a long history in monolingual parsing (Petrov et al., 2006; Liang et al., 2007; Cohen et al., 2014), where it reliably yields state- of-the-art phrase structure parsers based on gen- erative models, but we are the first to apply it to translation. We first generalize the concept of latent PCFGs to latent-variable SCFGs (§2). We then follow by a presentation of the tensor-based formulation …

Content truncated to 500 words