What is Hidden among Translation Rules

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 839–844, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics What is Hidden among Translation Rules Libin Shen Persado 50 West 17th Street New York, NY 10011 libin.shen@persado.com Bowen Zhou IBM T. J. Watson Research Center 1101 Kitchawan Road Yorktown Heights, NY 10598 zhou@us.ibm.com Abstract Most of the machine translation systems rely on a large set of translation rules. These rules are treated as discrete and independent events. In this short paper, we propose a novel method to model rules as observed generation output of a compact hidden model, which leads to better generalization capability. We present a preliminary generative model to test this idea. Experimental results show about one point im- provement on TER-BLEU over a strong base- line in Chinese-to-English translation. 1 Introduction Most of the modern Statistical Machine Transla- tion (SMT) systems, for example (Koehn et al., 2003; Och and Ney, 2004; Chiang, 2005; Marcu et al., 2006; Shen et al., 2008), employ a large rule set that may contain tens of millions of translation rules or even more. In these systems, each transla- tion rule has about 20 dense features, which repre- sent key statistics collected from the training data, such as word translation probability, phrase transla- tion probability etc. Except for these common fea- tures, there is no connection among the translation rules. The translation rules are treated as indepen- dent events. The use of sparse features as in (Arun and Koehn, 2007; Watanabe et al., 2007; Chiang et al., 2009) to some extent mitigated this problem. In their work, there are as many as 10,000 features defined on the appearance of certain frequent words and Part of Speech (POS) tags in rules. They provide signifi- cant improvement in automatic evaluation metrics. However, these sparse features fire quite randomly and infrequently on each rule. Thus, there is still plenty of space to better model translation rules. In this paper, we will explore the relationship among translation rules. We no longer view rules as discrete or unrelated events. Instead, we view rules, which are observed from training data, as ran- dom variables generated by a hidden model. This generative process itself is also hidden. All possible generative processes can be represented with factor- ized structures such as weighted hypergraphs and fi- nite state machines. This approach leads to a com- pact model that has better generalization capability and allows translation rules not explicitly observed in training date. This paper reports work-in-progress to exploit hidden relations among rules. Preliminary experi- ments show about one point improvement on TER- BLEU over a strong baseline in Chinese-to-English translation. 2 Hidden Models Let G = {(r, f)} be a grammar observed from paral- lel training data, where f is the frequency of a bilin- gual translation rule r. Let M be a hidden model that generates every translation rule r. For example, M could be mod- eled with a weighted hypergraph or finite state ma- …

Content truncated to 500 words