A Continuous Space Rule Selection Model for Syntax-based Statistical Machine Translation

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1372–1381, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A Continuous Space Rule Selection Model for Syntax-based Statistical Machine Translation Jingyi Zhang1,2, Masao Utiyama1, Eiichro Sumita1 Graham Neubig2, Satoshi Nakamura2 1National Institute of Information and Communications Technology, 3-5Hikaridai, Keihanna Science City, Kyoto 619-0289, Japan 2Graduate School of Information Science, Nara Institute of Science and Technology, Takayama, Ikoma, Nara 630-0192, Japan jingyizhang/mutiyama/eiichiro.sumita@nict.go.jp neubig/s-nakamura@is.naist.jp Abstract One of the major challenges for statisti- cal machine translation (SMT) is to choose the appropriate translation rules based on the sentence context. This paper pro- poses a continuous space rule selection (CSRS) model for syntax-based SMT to perform this context-dependent rule selec- tion. In contrast to existing maximum en- tropy based rule selection (MERS) mod- els, which use discrete representations of words as features, the CSRS model is learned by a feed-forward neural network and uses real-valued vector representa- tions of words, allowing for better gen- eralization. In addition, we propose a method to train the rule selection models only on minimal rules, which are more fre- quent and have richer training data com- pared to non-minimal rules. We tested our model on different translation tasks and the CSRS model outperformed a base- line without rule selection and the previ- ous MERS model by up to 2.2 and 1.1 points of BLEU score respectively. 1 Introduction In syntax-based statistical machine translation (SMT), especially tree-to-string (Liu et al., 2006; Graehl and Knight, 2004) and forest-to-string (Mi et al., 2008) SMT, a source tree or forest is used as input and translated by a series of tree-based trans- lation rules into a target sentence. A tree-based translation rule can perform reordering and trans- lation jointly by projecting a source subtree into a target string, which can contain both terminals and nonterminals. One of the difficulties in applying this model is the ambiguity existing in translation rules: a S NP a thief caught PRP VBD DT NN NP VP I 我抓了 一个 贼 S NP a cold caught PRP VBD DT NN NP VP I 得了 感冒 我 S NP caught VBD NP VP x0 了 x1 抓 x0 x1 S NP caught VBD NP VP x0 了 x1 得 x0 x1 Rule Extraction Figure 1: An ambiguous source subtree with dif- ferent translations (English-to-Chinese). source subtree can have different target transla- tions extracted from the parallel corpus as shown in Figure 1. Selecting correct rules during decod- ing is a major challenge for SMT in general, and syntax-based models are no exception. There have been several methods proposed to resolve this ambiguity. The most simple method, used in the first models of tree-to-string transla- tion (Liu et al., 2006), estimated the probability of a translation rule by relative frequencies. For ex- ample, in Figure 1, the rule that occurs more times in the training data will have a higher score. Later, Liu et al. (2008) proposed a maximum entropy based rule selection (MERS, …

Content truncated to 500 words