Leave-one-out Word Alignment without Garbage Collector Effects

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1817–1827, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Leave-one-out Word Alignment without Garbage Collector Effects Xiaolin Wang Masao Utiyama Andrew Finch Taro Watanabe∗ Eiichiro Sumita Advanced Translation Research and Development Promotion Center National Institute of Information and Communications Technology, Japan {xiaolin.wang,mutiyama,andrew.finch,eiichiro.sumita}@nict.go.jp tarow@google.com Abstract Expectation-maximization algorithms, such as those implemented in GIZA++ pervade the field of unsupervised word alignment. However, these algorithms have a problem of over-fitting, leading to “garbage collector effects,” where rare towords tend to be erroneously aligned untranslated words. This paper proposes a leave-one-out expectation- maximization algorithm for unsupervised word alignment to address this prob- lem. The proposed method excludes information derived from the alignment of a sentence pair from the alignment models used to align it. This prevents erroneous alignments within a sentence pair from supporting themselves. Ex- perimental results on Chinese-English and Japanese-English corpora show that the F1, precision and recall of alignment were consistently increased by 5.0% – 17.2%, and BLEU scores of end-to-end translation were raised by 0.03 – 1.30. The proposed method also outperformed l0-normalized GIZA++ and Kneser-Ney smoothed GIZA++. 1 Introduction Unsupervised word alignment (WA) on bilingual sentence pairs serves as an essential foundation for building most statistical machine translation (SMT) systems. A lot of methods have been pro- posed to raise the accuracy of WA in an effort to improve end-to-end translation quality. This pa- per contributes to this effort through refining the widely used expectation-maximization (EM) algo- rithm for WA (Dempster et al., 1977; Brown et al., 1993b; Och and Ney, 2000). ∗ The author now is affiliated with Google, Japan. The EM algorithm for WA has a great influ- ence in SMT. Many well-known toolkits includ- ing GIZA++ (Och and Ney, 2003), the Berkeley Aligner (Liang et al., 2006; DeNero and Klein, 2007), Fast Align (Dyer et al., 2013) and SyM- GIZA++ (Junczys-Dowmunt and Sza, 2012), all employ this algorithm. GIZA++ in particular is frequently used in systems participating in many shared tasks (Goto et al., 2011; Cettolo et al., 2013; Bojar et al., 2013). However, the EM algorithm for WA is well- known for introducing “garbage collector ef- fects.” Rare words have a tendency to collect garbage, that is they have a tendency to be erro- neously aligned to untranslated words (Brown et al., 1993a; Moore, 2004; Ganchev et al., 2008; V Grac¸a et al., 2010). Figure 1(a) shows a real sentence pair, denoted s, from the GALE Chinese- English Word Alignment and Tagging Training corpus (GALE WA corpus)1 with it’s human- annotated word alignment. The Chinese word “HE ZHANG,” denoted wr, which means river custodian, only occurs once in the whole corpus. We performed EM training using GIZA++ on this corpus concatenated with 442,967 training sen- tence pairs from the NIST Open Machine Trans- lation (OpenMT) 2006 evaluation2. The resulting alignment is shown in Figure 1(b). It can be seen that wr is erroneously aligned to multiple English words. To find …

Content truncated to 500 words