Generalized Agreement for Bidirectional Word Alignment

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1828–1836, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Generalized Agreement for Bidirectional Word Alignment Chunyang Liu†, Yang Liu†∗, Huanbo Luan†, Maosong Sun†, and Heng Yu‡ † State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China ‡ Samsung R&D Institute of China, Beijing 100028, China {liuchunyang2012,liuyang.china,luanhuanbo}@gmail.com, sms@tsinghua.edu.cn h0517.yu@samsung.com Abstract While agreement-based joint training has proven to deliver state-of-the-art align- ment accuracy, the produced word align- ments are usually restricted to one-to- one mappings because of the hard cons- traint on agreement. We propose a ge- neral framework to allow for arbitrary loss functions that measure the disagreement between asymmetric alignments. The loss functions can not only be defined between asymmetric alignments but al- so between alignments and other latent structures such as phrase segmentations. We use a Viterbi EM algorithm to train the joint model since the inference is intractable. Experiments on Chinese- English translation show that joint training with generalized agreement achieves sig- nificant improvements over two state-of- the-art alignment methods. 1 Introduction Word alignment is a natural language process- ing task that aims to specify the correspondence between words in two languages (Brown et al., 1993). It plays an important role in statistical machine translation (SMT) as word-aligned bi- lingual corpora serve as the input of translation rule extraction (Koehn et al., 2003; Chiang, 2007; Galley et al., 2006; Liu et al., 2006). Although state-of-the-art generative alignment models (Brown et al., 1993; Vogel et al., 1996) have been widely used in practical SMT systems, they fail to model the symmetry of word align- ment. While word alignments in real-world bi- lingual data usually exhibit complicated mappings (i.e., mixed with one-to-one, one-to-many, many- to-one, and many-to-many links), these models as- sume that each target word is aligned to exactly ∗Corresponding author: Yang Liu. one source word. To alleviate this problem, heuris- tic methods (e.g., grow-diag-final) have been pro- posed to combine two asymmetric alignments (source-to-target and target-to-source) to generate symmetric bidirectional alignments (Och and Ney, 2003; Koehn and Hoang, 2007). Instead of using heuristic symmetrization, Liang et al. (2006) introduce a principled approach that encourages the agreement between asymmetric alignments in two directions. The basic idea is to favor links on which both uni- directional models agree. They associate two models via the agreement constraint and show that agreement-based joint training improves align- ment accuracy significantly. However, enforcing agreement in joint training faces a major problem: the two models are restrict- ed to one-to-one alignments (Liang et al., 2006). This significantly limits the translation accuracy, especially for distantly-related language pairs such as Chinese-English (see Section 5). Although pos- terior decoding can potentially address this prob- lem, Liang et al. (2006) find that many-to-many alignments occur infrequently because posteriors are sharply peaked around the Viterbi alignments. We believe that this happens because their model imposes a hard …

Content truncated to 500 words