A pilot study towards end-to-end MT training

A Pilot Study Towards End-to-End MT Training Feifei Zhai ffzhai2012@gmail.com Queens College, City University of New York, Queens, NY 11367 Liang Huang∗ lianghuang.sh@gmail.com School of EECS, Oregon State University, Corvallis, OR 97330 Abstract Typical MT training involves several stages, including word alignment, rule extraction, trans- lation model estimation, and parameter tuning. In this paper, different from the traditional pipeline, we investigate the possibility of end-to-end MT training, and propose a framework which combines rule induction and parameter tuning in one single module. Preliminary exper- iments show that our learned model achieves comparable translation quality to the traditional MT training pipeline. 1 Introduction Typically, as shown in Figure 1(a), traditional machine translation (MT) training involves a long pipeline of several stages, including word alignment (by GIZA++), rule extraction, translation model (TM) estimation (by max-likelihood), and parameter tuning by MERT (Och, 2003), PRO (Hopkins and May, 2011), or MIRA (Watanabe et al., 2007; Chiang et al., 2008). This cascaded procedure inevitably propagates errors downstream, while at the same time making MT training overly complicated. rule extr. TM est. tune hard word alignment rule set rule probs full model GIZA (a) bitext full model (b) bitext joint framework for rule induction & parameter tuning soft align probs full model GIZA (c) bitext ideal framework for rule induction & parameter tuning Figure 1: Three approaches to MT training. (a) the standard pipeline; (b) ideal end-to-end MT training; (c) our work, in between (a) and (b), combines rule induction, TM estimation, and parameter tuning in one module. In this paper, instead of following the traditional training pipeline, we explore the pos- sibility of end-to-end MT training, which would ideally induce a full model with translation ∗Work done while Prof. Liang Huang was in City University of New York. Proceedings of MT Summit XV, vol.1: MT Researchers' Track Miami, Oct 30 - Nov 3, 2015 | p. 56 rules at the same time (as shown in Figure 1 (b)). Practically, it is a big challenge, because the search space would be too prohibitive if we consider all possible rules from scratch without any constraints from word alignment. To make this idea computationally affordable, we take a step back and propose a joint learning framework to do rule induction and parameter tuning together, shown in Figure 1 (c). Instead of knowing nothing at the beginning, we use word translation probabilities from GIZA++ to compute the lexical translation probabilities of phrase pairs, which is an effective feature guiding us to select good translation rules. Note that even with these probabilities, this is still a non-trival problem, since the joint learner still needs to deal with the entire space of all possible translation rules and all possible decoding derivations. For simplicity reasons we use phrase-based translation. Final experiments show that the joint learned model achieves comparable performance to the conventional training pipeline in a small scale. To our best knowledge, this is the first time, although on small data, verifying that it is possible to do effective end-to-end MT …

Content truncated to 500 words