Transliteration Alignment

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 136–144, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP Transliteration Alignment Vladimir Pervouchine, Haizhou Li Institute for Infocomm Research A*STAR, Singapore 138632 {vpervouchine,hli}@i2r.a-star.edu.sg Bo Lin School of Computer Engineering NTU, Singapore 639798 linbo@pmail.ntu.edu.sg Abstract This paper studies transliteration align- ment, its evaluation metrics and applica- tions. We propose a new evaluation met- ric, alignment entropy, grounded on the information theory, to evaluate the align- ment quality without the need for the gold standard reference and compare the metric with F-score. We study the use of phono- logical features and affinity statistics for transliteration alignment at phoneme and grapheme levels. The experiments show that better alignment consistently leads to more accurate transliteration. In transliter- ation modeling application, we achieve a mean reciprocal rate (MRR) of 0.773 on Xinhua personal name corpus, a signifi- cant improvement over other reported re- sults on the same corpus. In transliteration validation application, we achieve 4.48% equal error rate on a large LDC corpus. 1 Introduction Transliteration is a process of rewriting a word from a source language to a target language in a different writing system using the word’s phono- logical equivalent. The word and its translitera- tion form a transliteration pair. Many efforts have been devoted to two areas of studies where there is a need to establish the correspondence between graphemes or phonemes between a transliteration pair, also known as transliteration alignment. One area is the generative transliteration model- ing (Knight and Graehl, 1998), which studies how to convert a word from one language to another us- ing statistical models. Since the models are trained on an aligned parallel corpus, the resulting statisti- cal models can only be as good as the alignment of the corpus. Another area is the transliteration vali- dation, which studies the ways to validate translit- eration pairs. For example Knight and Graehl (1998) use the lexicon frequency, Qu and Grefen- stette (2004) use the statistics in a monolingual corpus and the Web, Kuo et al. (2007) use proba- bilities estimated from the transliteration model to validate transliteration candidates. In this paper, we propose using the alignment distance between the a bilingual pair of words to establish the evi- dence of transliteration candidacy. An example of transliteration pair alignment is shown in Figure 1. e5 e1 e2 e3 e4 c1 c2 c3 A L I C E 艾 丽 斯 source graphemes target graphemes e1 e2 e3 grapheme tokens Figure 1: An example of grapheme alignment (Al- ice, 艾丽斯), where a Chinese grapheme, a char- acter, is aligned to an English grapheme token. Like the word alignment in statistical ma- chine translation (MT), transliteration alignment becomes one of the important topics in machine transliteration, which has several unique chal- lenges. Firstly, the grapheme sequence in a word is not delimited into grapheme tokens, resulting in an additional level of complexity. Secondly, to maintain the phonological equivalence, the align- ment has to …

Content truncated to 500 words