Combining Clues for Word Alignment

Combining Clues for Word Alignment Rirg Tiedemann Department of Linguistics Uppsala University Box 527 SE-751 20 Uppsala, Sweden joerg@stp.ling.uu.se Abstract In this paper, a word alignment ap- proach is presented which is based on a combination of clues. Word align- ment clues indicate associations be- tween words and phrases. They can be based on features such as frequency, part-of-speech, phrase type, and the ac- tual wordform strings. Clues can be found by calculating similarity mea- sures or learned from word aligned data. The clue alignment approach, which is proposed in this paper, makes it possi- ble to combine association clues taking different kinds of linguistic information into account. It allows a dynamic to- kenization into token units of varying size. The approach has been applied to an English/Swedish parallel text with promising results. 1 Introduction Parallel corpora carry a huge amount of bilingual lexical information. Word alignment approaches focus on the automatic identification of translation relations in translated texts. Alignments are usu- ally represented as a set of links between words and phrases of source and target language seg- ments. An alignment can be complete, i.e. all items in both segments have been linked to cor- responding items in the other language, or incom- plete, otherwise. Alignments may include "null links" which can be modeled as links to an "empty element". In word alignment, we have to • find an appropriate model M for the align- ment of source and target language texts (modeling) • estimate parameters of the model M, e.g. from empirical data (parameter estimation) • find the optimal alignment of words and phrases for a given translation according to the model M and its parameters (alignment recovery). Modeling the relations between lexical units of translated texts is not a trivial task due to the di- versity of natural languages. There are generally two approaches, the estimation approach which is used in, e.g., statistical machine translation, and the association approach which is used in, e.g., automatic extraction of bilingual terminology. In the estimation approach, alignment parameters are modeled as hidden parameters in a statistical trans- lation model (Och and Ney, 2000). Association approaches base the alignment on similarity mea- sures and association tests such as Dice scores (Smadj a et al., 1996; Tiedemann, 1999), t-scores (Ahrenberg et al., 1998) log-likelihood measures (Tufis and Barbu, 2002), and longest common sub- sequence ratios (Melamed, 1995). One of the main difficulties in all alignment strategies is the identification of appropriate units in the source and the target language to be aligned. This task is hard even for human experts as can be seen in the detailed guidelines which are required 339 for manual alignments (Merkel, 1999; Melamed, 1998). Many translation relations involve multi- word units such as phrasal compounds, idiomatic expressions, and complex terms. Syntactic shifts can also require the consideration of a context larger than a single word. Some items are not translated at all. Splitting source and target language texts into appropriate units for align- ment (henceforth: tokenization) is …

Content truncated to 500 words