A Corpus Level MIRA Tuning Strategy for Machine Translation

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 851–856, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics A Corpus Level MIRA Tuning Strategy for Machine Translation Ming Tan, Tian Xia, Shaojun Wang Wright State University 3640 Colonel Glenn Hwy, Dayton, OH 45435 USA {tan.6, xia.7, shaojun.wang} @wright.edu Bowen Zhou IBM T.J. Watson Research Center 1101 Kitchawan Rd, Yorktown Heights, NY 10598 USA zhou@us.ibm.com Abstract MIRA based tuning methods have been widely used in statistical machine translation (SMT) system with a large number of fea- tures. Since the corpus-level BLEU is not de- composable, these MIRA approaches usually define a variety of heuristic-driven sentence- level BLEUs in their model losses. Instead, we present a new MIRA method, which em- ploys an exact corpus-level BLEU to com- pute the model loss. Our method is simpler in implementation. Experiments on Chinese-to- English translation show its effectiveness over two state-of-the-art MIRA implementations. 1 Introduction Margin infused relaxed algorithm (MIRA) has been widely adopted for the parameter optimization in SMT with a large feature size (Watanabe et al., 2007; Chiang et al., 2008; Chiang et al., 2009; Chiang, 2012; Eidelman, 2012; Cherry and Foster, 2012). Since BLEU is defined on the corpus, and not de- composed into sentences, most MIRA approaches consider a variety of sentence-level BLEUs for the model losses, many of which are heuristic-driven (Watanabe et al., 2007; Chiang et al., 2008; Chi- ang et al., 2009; Chiang, 2012; Cherry and Foster, 2012). The sentence-level BLEU appearing in the objective is generally based on a pseudo-document, which may not precisely reflect the corpus-level BLEU. We believe that this mismatch could poten- tially harm the performance. To avoid the sentence BLEU, the work in (Haddow et al., 2011) proposed to process sentences in small batches. The authors adopted a Gibbs sampling (Arun et al., 2009) tech- nique to search the hope and fear hypotheses, and they did not compare with MIRA. Watanabe (2012) also tuned the parameters with small batches of sen- tences and optimized a hinge loss not explicitly re- lated to BLEU using stochastic gradient descent. Both approaches introduced additional complexities over baseline MIRA approaches. In contrast, we propose a remarkably simple but efficient batch MIRA approach which exploits the exact corpus-level BLEU to compute model losses. We search for a hope and a fear hypotheses for the corpus with a straightforward approach and mini- mize the structured hinge loss defined on them. The experiments show that our method consistently out- performs two state-of-the-art MIRAs in Chinese-to- English translation tasks with a moderate margin. 2 Margin Infused Relaxed Algorithm We optimize the model parameters based on N-best lists. Our development (dev) set is a set of triples {(fi, ei, ri)}M i=1, where fi is a source-language sen- tence, corresponded by a list of target-language hy- potheses ei = {eij}N(fi) j=1 , with a number of refer- ences ri. h(eij ) is a feature vector. Generally, most decoders return a top-1 candidate as the transla- …

Content truncated to 500 words