Unsupervised Word Alignment Using Frequency Constraint in Posterior Regularized EM

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 153–158, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Unsupervised Word Alignment Using Frequency Constraint in Posterior Regularized EM Hidetaka Kamigaito1,2, Taro Watanabe2, Hiroya Takamura1, Manabu Okumura1 1Tokyo Institute of Technology, Precision and Intelligence Laboratory 4259 Nagatsuta-cho Midori-ku Yokohama, Japan 2National Institute of Information and Communication Technology 3-5 Hikari-dai, Seika-cho, Soraku-gun, Kyoto, Japan Abstract Generative word alignment models, such as IBM Models, are restricted to one- to-many alignment, and cannot explicitly represent many-to-many relationships in a bilingual text. The problem is par- tially solved either by introducing heuris- tics or by agreement constraints such that two directional word alignments agree with each other. In this paper, we fo- cus on the posterior regularization frame- work (Ganchev et al., 2010) that can force two directional word alignment models to agree with each other during train- ing, and propose new constraints that can take into account the difference between function words and content words. Ex- perimental results on French-to-English and Japanese-to-English alignment tasks show statistically significant gains over the previous posterior regularization baseline. We also observed gains in Japanese-to- English translation tasks, which prove the effectiveness of our methods under gram- matically different language pairs. 1 Introduction Word alignment is an important component in sta- tistical machine translation (SMT). For instance phrase-based SMT (Koehn et al., 2003) is based on the concept of phrase pairs that are automat- ically extracted from bilingual data and rely on word alignment annotation. Similarly, the model for hierarchical phrase-based SMT is built from exhaustively extracted phrases that are, in turn, heavily reliant on word alignment. The Generative word alignment models, such as the IBM Models (Brown et al., 1993) and HMM (Vogel et al., 1996), are popular methods for au- tomatically aligning bilingual texts, but are re- stricted to represent one-to-many correspondence of each word. To resolve this weakness, vari- ous symmetrization methods are proposed. Och and Ney (2003) and Koehn et al. (2003) propose various heuristic methods to combine two direc- tional models to represent many-to-many relation- ships. As an alternative to heuristic methods, fil- tering methods employ a threshold to control the trade-off between precision and recall based on a score estimated from the posterior probabili- ties from two directional models. Matusov et al. (2004) proposed arithmetic means of two mod- els as a score for the filtering, whereas Liang et al. (2006) reported better results using geometric means. The joint training method (Liang et al., 2006) enforces agreement between two directional models. Posterior regularization (Ganchev et al., 2010) is an alternative agreement method which directly encodes agreement during training. DeN- ero and Macherey (2011) and Chang et al. (2014) also enforce agreement during decoding. However, these agreement models do not take into account the difference in language pairs, which is crucial for linguistically different lan- guage pairs, such as Japanese and English: al- though content words may be aligned with each other by introducing some agreement constraints, …

Content truncated to 500 words