Guiding Statistical Word Alignment Models With Prior Knowledge

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 1–8, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Guiding Statistical Word Alignment Models With Prior Knowledge Yonggang Deng and Yuqing Gao IBM T. J. Watson Research Center Yorktown Heights, NY 10598 {ydeng,yuqing}@us.ibm.com Abstract We present a general framework to incor- porate prior knowledge such as heuristics or linguistic features in statistical generative word alignment models. Prior knowledge plays a role of probabilistic soft constraints between bilingual word pairs that shall be used to guide word alignment model train- ing. We investigate knowledge that can be derived automatically from entropy princi- ple and bilingual latent semantic analysis and show how they can be applied to im- prove translation performance. 1 Introduction Statistical word alignment models learn word as- sociations between parallel sentences from statis- tics. Most models are trained from corpora in an unsupervised manner whose success is heavily de- pendent on the quality and quantity of the training data. It has been shown that human knowledge, in the form of a small amount of manually anno- tated parallel data to be used to seed or guide model training, can significantly improve word alignment F-measure and translation performance (Ittycheriah and Roukos, 2005; Fraser and Marcu, 2006). As formulated in the competitive linking algo- rithm (Melamed, 2000), the problem of word align- ment can be regarded as a process of word link- age disambiguation, that is, choosing correct asso- ciations among all competing hypothesis. The more reasonable constraints are imposed on this process, the easier the task would become. For instance, the most relaxed IBM Model-1, which assumes that any source word can be generated by any target word equally regardless of distance, can be improved by demanding a Markov process of alignments as in HMM-based models (Vogel et al., 1996), or imple- menting a distribution of number of target words linked to a source word as in IBM fertility-based models (Brown et al., 1993). Following the path, we shall put more constraints on word alignment models and investigate ways of implementing them in a statistical framework. We have seen examples showing that names tend to align to names and function words are likely to be linked to function words. These observations are independent of language and can be understood by common sense. Moreover, there are other linguis- tically motivated constraints. For instance, words aligned to each other presumably are semantically consistent; and likely to be, they are syntactically agreeable. In these paper, we shall exploit some of these constraints in building better word alignments in the application of statistical machine translation. We propose a simple framework that can inte- grate prior knowledge into statistical word align- ment model training. In the framework, prior knowl- edge serves as probabilistic soft constraints that will guide word alignment model training. We present two types of constraints that are derived in an un- supervised way: one is based on the entropy prin- ciple, the other comes from bilingual latent …

Content truncated to 500 words