Scalable Variational Inference for Extracting Hierarchical Phrase-based Translation Rules

International Joint Conference on Natural Language Processing, pages 438–446, Nagoya, Japan, 14-18 October 2013. Scalable Variational Inference for Extracting Hierarchical Phrase-based Translation Rules∗ Baskaran Sankaran Simon Fraser University Burnaby BC. Canada baskaran@cs.sfu.ca Gholamreza Haffari Monash University Clayton VIC. Australia reza@monash.edu Anoop Sarkar Simon Fraser University Burnaby BC. Canada anoop@cs.sfu.ca Abstract We present a Variational-Bayes model for learning rules for the Hierarchical phrase- based model directly from the phrasal alignments. Our model is an alternative to heuristic rule extraction in hierarchical phrase-based translation (Chiang, 2007), which uniformly distributes the probabil- ity mass to the extracted rules locally. In contrast, in our approach the probability assigned to a rule is globally determined by its contribution towards all phrase pairs and results in a sparser rule set. We also propose a distributed framework for efficiently running inference for realistic MT corpora. Our experiments translating Korean, Arabic and Chinese into English demonstrate that they are able to exceed or retain the performance of baseline hier- archical phrase-based models. 1 Introduction Hierarchical phrase-based translation (Hiero) as described in (Chiang, 2005; Chiang, 2007) uses a synchronous context-free grammar (SCFG) de- rived from heuristically extracted phrase pairs obtained by symmetrizing bidirectional many-to- many word alignments (Och and Ney, 2004). The phrase-pairs are constrained by the source-target alignments such that all the alignment links from the source (target) words are connected to the target (source) words within the phrase. Given a word-aligned sentence pair ⟨fJ 1 , eI 1, A⟩, where A indicate the alignments, the source-target se- quence pair ⟨fj i , ej′ i′ ⟩ can be a phrase-pair iff the following alignment constraint is satisfied. (k, k′) ∈ A : k ∈ [i, j] ⇔ k′ ∈ [i′, j′] Given the phrase-pairs, SCFG rules are extracted by replacing aligned sequences of words in source ∗This research was partially supported by an NSERC, Canada (RGPIN: 264905) grant and a Google Faculty Award to the third author. and target sides by co-indexed non-terminals and rewriting the replaced source-target word se- quences as separate rules. Consider a rule X → ⟨β, γ⟩, where β and γ are sequences of termi- nals and non-terminals. Now, given another rule X → ⟨fj i , ej′ i′ ⟩, such that fj i and ej′ i′ are contained fully within β and γ as sub-phrases, the larger rule could be rewritten to create a new rule. X → ⟨βpXkβs, γpXkγs⟩ (1) Here βp (βs) refers to any prefix (suffix) of β that precedes (follows) fj i . Note that the non-terminals are co-indexed with a unique index so that they are rewritten simultaneously. 数 月 , 联合国 难民 专员 公署 months , the unhcr Figure 1: Chinese-English phrase-pair with align- ments As a concrete example, consider the word aligned Chinese-English phrase pair shown in Fig- ure 1. Notice that the phrase 联合国 (united na- tions) is incorrectly aligned to English determiner the, even though in ideal case the entire Chinese phrase 联合国 难民 专员 公署 should be aligned on the English side to …

Content truncated to 500 words