Semi-Supervised Training for Statistical Word Alignment

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 769–776, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Semi-Supervised Training for Statistical Word Alignment Alexander Fraser ISI / University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292 fraser@isi.edu Daniel Marcu ISI / University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292 marcu@isi.edu Abstract We introduce a semi-supervised approach to training for statistical machine transla- tion that alternates the traditional Expecta- tion Maximization step that is applied on a large training corpus with a discriminative step aimed at increasing word-alignment quality on a small, manually word-aligned sub-corpus. We show that our algorithm leads not only to improved alignments but also to machine translation outputs of higher quality. 1 Introduction The most widely applied training procedure for statistical machine translation — IBM model 4 (Brown et al., 1993) unsupervised training fol- lowed by post-processing with symmetrization heuristics (Och and Ney, 2003) — yields low quality word alignments. When compared with gold standard parallel data which was manually aligned using a high-recall/precision methodology (Melamed, 1998), the word-level alignments pro- duced automatically have an F-measure accuracy of 64.6 and 76.4% (see Section 2 for details). In this paper, we improve word alignment and, subsequently, MT accuracy by developing a range of increasingly sophisticated methods: 1. We first recast the problem of estimating the IBM models (Brown et al., 1993) in a dis- criminative framework, which leads to an ini- tial increase in word-alignment accuracy. 2. We extend the IBM models with new (sub)models, which leads to additional in- creases in word-alignment accuracy. In the process, we also show that these improve- ments are explained not only by the power of the new models, but also by a novel search procedure for the alignment of highest prob- ability. 3. Finally, we propose a training procedure that interleaves discriminative training with max- imum likelihood training. These steps lead to word alignments of higher accuracy which, in our case, correlate with higher MT accuracy. The rest of the paper is organized as follows. In Section 2, we review the data sets we use to validate experimentally our algorithms and the as- sociated baselines. In Section 3, we present itera- tively our contributions that eventually lead to ab- solute increases in alignment quality of 4.8% for French/English and 4.8% for Arabic/English, as measured using F-measure for large word align- ment tasks. These contributions pertain to the casting of the training procedure in the discrim- inative framework (Section 3.1); the IBM model extensions and modified search procedure for the Viterbi alignments (Section 3.2); and the in- terleaved, minimum error/maximum likelihood, training algorithm (Section 4). In Section 5, we as- sess the impact that our improved alignments have on MT quality. We conclude with a comparison of our work with previous research on discriminative training for word alignment and a short discussion of semi-supervised learning. 2 Data Sets and Baseline We conduct …

Content truncated to 500 words