Direct Orthographical Mapping for Machine Transliteration

Direct Orthographical Mapping for Machine Transliteration ZHANG Min LI Haizhou SU Jian Institute for Infocomm Research 21 Heng Mui Keng Terrace, Singapore 11961 {mzhang, hli, sujian}@i2r.a-star.edu.sg Abstract Machine transliteration/back-transliteration plays an important role in many multilingual speech and language applications. In this paper, a novel framework for machine transliteration/back- transliteration that allows us to carry out direct orthographical mapping (DOM) between two different languages is presented. Under this framework, a joint source-channel transliteration model, also called n-gram transliteration model (n- gram TM), is further proposed to model the transliteration process. We evaluate the proposed methods through several transliteration/back- transliteration experiments for English/Chinese and English/Japanese language pairs. Our study reveals that the proposed method not only reduces an extensive system development effort but also improves the transliteration accuracy significantly. 1 Introduction Many technical terms and proper names, such as personal, location and organization names, are translated from one language into another language with approximate phonetic equivalents. The phonetic translation from the native language to foreign language is defined as transliteration; conversely, the process of recalling a word in native language from a transliteration is defined as back-transliteration. For example, English name “Smith” and “史密斯 (pinyin 1: Shi-Mi-Si)” in Chinese form a pair of transliteration and back- transliteration. In many natural language processing tasks, such as multilingual named entity and term processing, machine translation, corpus alignment, cross lingual information retrieval and automatic bilingual dictionary compilation, automatic name transliteration has become an indispensable component. Recent efforts are reported for several language pairs, such as English/Chinese (Meng et al., 2001; Virga et al., 2003; Lee et al., 2003; Gao et al., 2004; Guo et al., 2004), English/Japanese (Knight et al., 1998; Brill et al., 2001; Bilac et al., 2004), 1 Pinyin is the standard Romanization of Chinese. English/Korean (Oh et al., 2002; Sung et al., 2000), and English/Arabic (Yaser et al., 2002). Most of the reported works utilize a phonetic clue to resolve the transliteration through a multiple step phonemic mapping where algorithms, such as dictionary lookup, rule-based and machine learning-based approaches, have been well explored. In this paper, we will discuss the limitation of the previous works and present a novel framework for machine transliteration. The new framework carries out the transliteration by direct orthographical mapping (DOM) without any intermediate phonemic mapping. Under this framework, we further propose a joint source- channel transliteration mode (n-gram TM) as an alternative machine learning-based approach to model the source-target word orthographic association. Without the loss of generality, we evaluate the performance of the proposed method for English/Chinese and English/Japanese pairs. An experiment that compares the proposed method with several state-of-art approaches is also presented. The results reveal that our method outperforms other previous methods significantly. The reminder of the paper is organized as follows. Section 2 reviews the previous work. In section 3, the DOM framework and n-gram TM model are formulated. Section 4 describes the evaluation results and compares our method with other reported work. Finally, we conclude the study with some discussions. 2 Previous …

Content truncated to 500 words