Collapsed Consonant and Vowel Models: New Approaches for English-Persian Transliteration and Back-Transliteration

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 648–655, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Collapsed Consonant and Vowel Models: New Approaches for English-Persian Transliteration and Back-Transliteration Sarvnaz Karimi Falk Scholer Andrew Turpin School of Computer Science and Information Technology RMIT University, GPO Box 2476V, Melbourne 3001, Australia {sarvnaz,fscholer,aht}@cs.rmit.edu.au Abstract We propose a novel algorithm for English to Persian transliteration. Previous meth- ods proposed for this language pair apply a word alignment tool for training. By contrast, we introduce an alignment algo- rithm particularly designed for translitera- tion. Our new model improves the English to Persian transliteration accuracy by 14% over an n-gram baseline. We also propose a novel back-transliteration method for this language pair, a previously unstudied prob- lem. Experimental results demonstrate that our algorithm leads to an absolute improve- ment of 25% over standard transliteration approaches. 1 Introduction Translation of a text from a source language to a target language requires dealing with technical terms and proper names. These occur in almost any text, but rarely appear in bilingual dictionar- ies. The solution is the transliteration of such out-of- dictionary terms: a word from the source language is transformed to a word in the target language, pre- serving its pronunciation. Recovering the original word from the transliterated target is called back- transliteration. Automatic transliteration is impor- tant for many different applications, including ma- chine translation, cross-lingual information retrieval and cross-lingual question answering. Transliteration methods can be categorized into grapheme-based (AbdulJaleel and Larkey, 2003; Li et al., 2004), phoneme-based (Knight and Graehl, 1998; Jung et al., 2000), and combined (Bilac and Tanaka, 2005) approaches. Grapheme-based meth- ods perform a direct orthographical mapping be- tween source and target words, while phoneme- based approaches use an intermediate phonetic rep- resentation. Both grapheme- or phoneme-based methods usually begin by breaking the source word into segments, and then use a source segment to tar- get segment mapping to generate the target word. The rules of this mapping are obtained by aligning already available transliterated word pairs (training data); alternatively, such rules can be handcrafted. From this perspective, past work is roughly divided into those methods which apply a word alignment tool such as GIZA++ (Och and Ney, 2003), and ap- proaches that combine the alignment step into their main transliteration process. Transliteration is language dependent, and meth- ods that are effective for one language pair may not work as well for another. In this paper, we investigate the English-Persian transliteration prob- lem. Persian (Farsi) is an Indo-European language, written in Arabic script from right to left, but with an extended alphabet and different pronunciation from Arabic. Our previous approach to English- Persian transliteration introduced the grapheme- based collapsed-vowel method, employing GIZA++ for source to target alignment (Karimi et al., 2006). We propose a new transliteration approach that ex- tends the collapsed-vowel method. To meet Per- sian language transliteration requirements, we also propose a novel alignment algorithm in our training stage, which makes use of …

Content truncated to 500 words