Hierarchical Incremental Adaptation for Statistical Machine Translation

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1059–1065, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Hierarchical Incremental Adaptation for Statistical Machine Translation Joern Wuebker Lilt Inc. joern@lilt.com Spence Green Lilt Inc. spence@lilt.com John DeNero Lilt Inc. john@lilt.com Abstract We present an incremental adaptation ap- proach for statistical machine translation that maintains a flexible hierarchical do- main structure within a single consistent model. Both weights and rules are updated incrementally on a stream of post-edits. Our multi-level domain hierarchy allows the sys- tem to adapt simultaneously towards local context at different levels of granularity, in- cluding genres and individual documents. Our experiments show consistent improve- ments in translation quality from all com- ponents of our approach. 1 Introduction Suggestions from a machine translation system can increase the speed and quality of professional hu- man translators (Guerberof, 2009; Plitt and Mas- selot, 2010; Green et al., 2013a, inter alia). How- ever, querying a single fixed model for all different documents fails to incorporate contextual informa- tion that can potentially improve suggestion quality. We describe a model architecture that adapts simul- taneously to multiple genres and individual docu- ments, so that translation suggestions are informed by two levels of contextual information. Our primary technical contribution is a hierarchi- cal adaptation technique for a post-editing scenario with incremental adaptation, in which users request translations of sentences in corpus order and pro- vide corrected translations of each sentence back to the system (Ortiz-Martínez et al., 2010). Our learning approach resembles Hierarchical Bayesian Domain Adaptation (Finkel and Manning, 2009), but updates both the model weights and translation rules in real time based on these corrected transla- tions (Mathur et al., 2013; Denkowski et al., 2014). Our adapted system can provide on-demand trans- lations for any genre and document to which it has ever been exposed, using weights and rules for do- mains associated with each translation request. Our weight adaptation is performed using a hier- archical extension to fast and adaptive online train- ing (Green et al., 2013b), a technique based on Ada- Grad (Duchi et al., 2011) and forward-backward splitting (Duchi and Singer, 2009) that can accu- rately set weights for both dense and sparse fea- tures (Green et al., 2014b). Rather than adjusting all weights based on each example, our extension adjusts offsets to a fixed baseline system. In this way, the system can adapt to multiple genres while preventing cross-genre contamination. In large-scale experiments, we adapt a multi- genre baseline system to patents, lectures, and news articles. Our experiments show that sparse mod- els, hierarchical updates, and rule adaptation all contribute consistent improvements. We observe quality gains in all genres, validating our hypothe- sis that document and genre context are important additional inputs to a machine translation system used for post-editing. 2 Background The log-linear appoach to statistical machine trans- lation models the predictive translation distribution p(e|f; w) directly in log-linear form (Och and Ney, 2004): p(e|f; w) = � r: src(r)=f tgt(r)=e 1 …

Content truncated to 500 words