Incremental Syntactic Language Models for Phrase-based Translation

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 620–631, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Incremental Syntactic Language Models for Phrase-based Translation Lane Schwartz Air Force Research Laboratory Wright-Patterson AFB, OH USA lane.schwartz@wpafb.af.mil Chris Callison-Burch Johns Hopkins University Baltimore, MD USA ccb@cs.jhu.edu William Schuler Ohio State University Columbus, OH USA schuler@ling.ohio-state.edu Stephen Wu Mayo Clinic Rochester, MN USA wu.stephen@mayo.edu Abstract This paper describes a novel technique for in- corporating syntactic knowledge into phrase- based machine translation through incremen- tal syntactic parsing. Bottom-up and top- down parsers typically require a completed string as input. This requirement makes it dif- ficult to incorporate them into phrase-based translation, which generates partial hypothe- sized translations from left-to-right. Incre- mental syntactic language models score sen- tences in a similar left-to-right fashion, and are therefore a good mechanism for incorporat- ing syntax into phrase-based translation. We give a formal definition of one such linear- time syntactic language model, detail its re- lation to phrase-based decoding, and integrate the model with the Moses phrase-based trans- lation system. We present empirical results on a constrained Urdu-English translation task that demonstrate a significant BLEU score im- provement and a large decrease in perplexity. 1 Introduction Early work in statistical machine translation viewed translation as a noisy channel process comprised of a translation model, which functioned to posit ad- equate translations of source language words, and a target language model, which guided the fluency of generated target language strings (Brown et al., This research was supported by NSF CAREER/PECASE award 0447685, NSF grant IIS-0713448, and the European Commission through the EuroMatrixPlus project. Opinions, in- terpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the sponsors or the United States Air Force. Cleared for public release (Case Number 88ABW-2010-6489) on 10 Dec 2010. 1990). Drawing on earlier successes in speech recognition, research in statistical machine trans- lation has effectively used n-gram word sequence models as language models. Modern phrase-based translation using large scale n-gram language models generally performs well in terms of lexical choice, but still often produces ungrammatical output. Syntactic parsing may help produce more grammatical output by better model- ing structural relationships and long-distance depen- dencies. Bottom-up and top-down parsers typically require a completed string as input; this requirement makes it difficult to incorporate these parsers into phrase-based translation, which generates hypothe- sized translations incrementally, from left-to-right.1 As a workaround, parsers can rerank the translated output of translation systems (Och et al., 2004). On the other hand, incremental parsers (Roark, 2001; Henderson, 2004; Schuler et al., 2010; Huang and Sagae, 2010) process input in a straightforward left-to-right manner. We observe that incremental parsers, used as structured language models, pro- vide an appropriate algorithmic match to incremen- tal phrase-based decoding. We directly integrate in- cremental syntactic parsing into phrase-based trans- lation. This approach re-exerts the role of the lan- guage model as a mechanism for encouraging syn- tactically fluent translations. The contributions of this work …

Content truncated to 500 words