Joshua 4.0: Packing, PRO, and Paraphrases

Proceedings of the 7th Workshop on Statistical Machine Translation, pages 283–291, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Joshua 4.0: Packing, PRO, and Paraphrases Juri Ganitkevitch1, Yuan Cao1, Jonathan Weese1, Matt Post2, and Chris Callison-Burch1 1Center for Language and Speech Processing 2Human Language Technology Center of Excellence Johns Hopkins University Abstract We present Joshua 4.0, the newest version of our open-source decoder for parsing-based statistical machine translation. The main con- tributions in this release are the introduction of a compact grammar representation based on packed tries, and the integration of our implementation of pairwise ranking optimiza- tion, J-PRO. We further present the exten- sion of the Thrax SCFG grammar extractor to pivot-based extraction of syntactically in- formed sentential paraphrases. 1 Introduction Joshua is an open-source toolkit1 for parsing-based statistical machine translation of human languages. The original version of Joshua (Li et al., 2009) was a reimplementation of the Python-based Hiero ma- chine translation system (Chiang, 2007). It was later extended to support grammars with rich syntactic labels (Li et al., 2010a). More recent efforts in- troduced the Thrax module, an extensible Hadoop- based extraction toolkit for synchronous context- free grammars (Weese et al., 2011). In this paper we describe a set of recent exten- sions to the Joshua system. We present a new com- pact grammar representation format that leverages sparse features, quantization, and data redundancies to store grammars in a dense binary format. This al- lows for both near-instantaneous start-up times and decoding with extremely large grammars. In Sec- tion 2 we outline our packed grammar format and 1joshua-decoder.org present experimental results regarding its impact on decoding speed, memory use and translation quality. Additionally, we present Joshua’s implementation of the pairwise ranking optimization (Hopkins and May, 2011) approach to translation model tuning. J-PRO, like Z-MERT, makes it easy to implement new metrics and comes with both a built-in percep- tron classifier and out-of-the-box support for widely used binary classifiers such as MegaM and Max- Ent (Daum´e III and Marcu, 2006; Manning and Klein, 2003). We describe our implementation in Section 3, presenting experimental results on perfor- mance, classifier convergence, and tuning speed. Finally, we introduce the inclusion of bilingual pivoting-based paraphrase extraction into Thrax, Joshua’s grammar extractor. Thrax’s paraphrase ex- traction mode is simple to use, and yields state-of- the-art syntactically informed sentential paraphrases (Ganitkevitch et al., 2011). The full feature set of Thrax (Weese et al., 2011) is supported for para- phrase grammars. An easily configured feature-level pruning mechanism allows to keep the paraphrase grammar size manageable. Section 4 presents de- tails on our paraphrase extraction module. 2 Compact Grammar Representation Statistical machine translation systems tend to per- form better when trained on larger amounts of bilin- gual parallel data. Using tools such as Thrax, trans- lation models and their parameters are extracted and estimated from the data. In Joshua, translation models are represented as synchronous context-free grammars (SCFGs). An SCFG is a collection of 283 rules {ri} that take the form: ri = Ci …

Content truncated to 500 words