Bilingual Structured Language Models for Statistical Machine Translation

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2398–2408, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Bilingual Structured Language Models for Statistical Machine Translation Ekaterina Garmash and Christof Monz Informatics Institute, University of Amsterdam Science Park 904, 1098 XH Amsterdam, The Netherlands {e.garmash,c.monz}@uva.nl Abstract This paper describes a novel target-side syntactic language model for phrase-based statistical machine translation, bilingual structured language model. Our approach represents a new way to adapt structured language models (Chelba and Jelinek, 2000) to statistical machine translation, and a first attempt to adapt them to phrase- based statistical machine translation. We propose a number of variations of the bilingual structured language model and evaluate them in a series of rescoring ex- periments. Rescoring of 1000-best transla- tion lists produces statistically significant improvements of up to 0.7 BLEU over a strong baseline for Chinese-English, but does not yield improvements for Arabic- English. 1 Introduction Many model components of competitive statisti- cal machine translation (SMT) systems are based on rather simplistic definitions with little linguis- tic grounding, which includes the definitions of phrase pairs, lexicalized reordering, and n-gram language models. However, earlier work has also shown that statistical MT can benefit from ad- ditional linguistically motivated models. Most prominent among the linguistically motivated ap- proaches are syntax-based MT systems which take into account the syntactic structure of sen- tences through CKY decoding and categorial la- bels (Zollmann and Venugopal, 2006; Shen et al., 2008). On the other hand, the commonly used phrase-based SMT approaches can also reap some of the benefits of using syntactic information by integrating linguistic components addressing spe- cific phenomena, such as Cherry (2008), Carpuat et al. (2010), Crego and Yvon (2010), Ge (2010), Xiang et al. (2011), Lerner and Petrov (2013), Garmash and Monz (2014). This paper is a contribution to the existing body of work on how syntactically motivated models help translation performance. We work with the phrase-based SMT (PBSMT) (Koehn et al., 2003) framework as the baseline system. Our choice is motivated by the fact that PBSMT is a conceptu- ally simple and therefore flexible framework. It is typically quite straightforward to integrate an ad- ditional model into the system. Also, PBSMT is the most widely used framework in the SMT re- search community, which ensures comparability of our results to other people’s work on the topic. There is a variety of ways syntax can be used in a PBSMT model. Typically a syntactic represen- tation of a source sentence is used to define con- straints on the order in which the decoder trans- lates it. For example, Cherry (2008) defines soft constraints based on the notion of syntactic cohe- sion (Section 2). Ge (2010) captures reordering patterns by defining soft constraints based on the currently translated word’s POS tag and the words structurally related to it. On the other hand, tar- get syntax is more challenging to use in PBSMT, since a target-side syntactic model does not have access to the whole target …

Content truncated to 500 words