Log-Linear Models for Word Alignment

Proceedings of the 43rd Annual Meeting of the ACL, pages 459–466, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Log-linear Models for Word Alignment Yang Liu , Qun Liu and Shouxun Lin Institute of Computing Technology Chinese Academy of Sciences No. 6 Kexueyuan South Road, Haidian District P. O. Box 2704, Beijing, 100080, China {yliu, liuqun, sxlin}@ict.ac.cn Abstract We present a framework for word align- ment based on log-linear models. All knowledge sources are treated as feature functions, which depend on the source langauge sentence, the target language sentence and possible additional vari- ables. Log-linear models allow statis- tical alignment models to be easily ex- tended by incorporating syntactic infor- mation. In this paper, we use IBM Model 3 alignment probabilities, POS correspon- dence, and bilingual dictionary cover- age as features. Our experiments show that log-linear models significantly out- perform IBM translation models. 1 Introduction Word alignment, which can be defined as an object for indicating the corresponding words in a parallel text, was first introduced as an intermediate result of statistical translation models (Brown et al., 1993). In statistical machine translation, word alignment plays a crucial role as word-aligned corpora have been found to be an excellent source of translation-related knowledge. Various methods have been proposed for finding word alignments between parallel texts. There are generally two categories of alignment approaches: statistical approaches and heuristic approaches. Statistical approaches, which depend on a set of unknown parameters that are learned from training data, try to describe the relationship between a bilin- gual sentence pair (Brown et al., 1993; Vogel and Ney, 1996). Heuristic approaches obtain word align- ments by using various similarity functions between the types of the two languages (Smadja et al., 1996; Ker and Chang, 1997; Melamed, 2000). The cen- tral distinction between statistical and heuristic ap- proaches is that statistical approaches are based on well-founded probabilistic models while heuristic ones are not. Studies reveal that statistical alignment models outperform the simple Dice coefficient (Och and Ney, 2003). Finding word alignments between parallel texts, however, is still far from a trivial work due to the di- versity of natural languages. For example, the align- ment of words within idiomatic expressions, free translations, and missing content or function words is problematic. When two languages widely differ in word order, finding word alignments is especially hard. Therefore, it is necessary to incorporate all useful linguistic information to alleviate these prob- lems. Tiedemann (2003) introduced a word alignment approach based on combination of association clues. Clues combination is done by disjunction of single clues, which are defined as probabilities of associa- tions. The crucial assumption of clue combination that clues are independent of each other, however, is not always true. Och and Ney (2003) proposed Model 6, a log-linear combination of IBM transla- tion models and HMM model. Although Model 6 yields better results than naive IBM models, it fails to include dependencies other than IBM models and HMM model. Cherry and Lin (2003) developed a 459 statistical model to …

Content truncated to 500 words