Hierarchical Back-off Modeling of Hiero Grammar based on Non-parametric Bayesian Model

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1217–1227, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Hierarchical Back-off Modeling of Hiero Grammar based on Non-parametric Bayesian Model Hidetaka Kamigaito1 kamigaito@lr.pi.titech.ac.jp Taro Watanabe2 tarow@google.com Hiroya Takamura1 takamura@pi.titech.ac.jp Manabu Okumura1 oku@pi.titech.ac.jp Eiichiro Sumita3 1Tokyo Institute of Technologyeiichiro.sumita@nict.go.jp 2Google Japan Inc. 3National Institute of Information and Communication Technology Abstract In hierarchical phrase-based machine translation, a rule table is automatically learned by heuristically extracting syn- chronous rules from a parallel corpus. As a result, spuriously many rules are extracted which may be composed of various incorrect rules. The larger rule table incurs more run time for decoding and may result in lower translation quality. To resolve the problems, we propose a hierarchical back-off model for Hiero grammar, an instance of a synchronous context free grammar (SCFG), on the basis of the hierarchical Pitman-Yor process. The model can extract a compact rule and phrase table without resorting to any heuristics by hierarchically backing off to smaller phrases under SCFG. Inference is efficiently carried out using two-step synchronous parsing of Xiao et al., (2012) combined with slice sampling. In our experiments, the proposed model achieved higher or at least comparable translation quality against a previous Bayesian model on various language pairs; German/French/Spanish/Japanese- English. When compared against heuristic models, our model achieved comparable translation quality on a full size German- English language pair in Europarl v7 corpus with significantly smaller grammar size; less than 10% of that for heuristic model. 1 Introduction Hierarchical phrase-based statistical machine translation (HPBSMT) (Chiang, 2007) is a popu- lar alternative to phrase-based SMT (PBSMT), in which synchronous context free grammar (SCFG) is used as the basis of the machine translation model. With HPBSMT, a restricted form of an SCFG, i.e., Hiero grammar, is usually used and is especially suited for linguistically divergent lan- guage pairs, such as Japanese and English. How- ever, a rule table, i.e., a synchronous grammar, may be composed of spuriously many rules with potential errors especially when it was automati- cally acquired from a parallel corpus. As a result, the increase in the rule table incurs a large amount of time for decoding and may result in lower trans- lation quality. Pruning a rule table either on the basis of signif- icance test (Johnson et al., 2007) or entropy (Ling et al., 2012; Zens et al., 2012) used in PBSMT can be easily applied for HPBSMT. However, these methods still rely on a heuristically determined threshold parameter. Bayesian SCFG methods (Blunsom et al., 2009) solve the spurious rule extraction problem by directly inducing a com- pact rule table from a parallel corpus on the basis of a non-parametric Bayesian model without any heuristics. Training for Bayesian SCFG models infers a derivation tree for each training instance, which demands the time complexity of O(|f|3|e|3) when we use dynamic programming SCFG bi- parsing (Wu, 1997). Gibbs sampling without bi- parsing (Levenberg et al., 2012) can avoid this problem, though the induced derivation …

Content truncated to 500 words