Finding Ideographic Representations of Japanese Names Written in Latin Script via Language Identification and Corpus Validation

Finding Ideographic Representations of Japanese Names Written in Latin Script via Language Identification and Corpus Validation Yan Qu Clairvoyance Corporation 5001 Baum Boulevard, Suite 700 Pittsburgh, PA 15213-1854, USA yqu@clairvoyancecorp.com Gregory Grefenstette∗∗∗∗ LIC2M/LIST/CEA 18, route du Panorama, BP 6 Fontenay-aux-Roses, 92265 France Gregory.Grefenstette@cea.fr Abstract Multilingual applications frequently involve dealing with proper names, but names are often missing in bilingual lexicons. This problem is exacerbated for applications involving translation between Latin-scripted languages and Asian languages such as Chinese, Japanese and Korean (CJK) where simple string copying is not a solution. We present a novel approach for generating the ideographic representations of a CJK name written in a Latin script. The proposed approach involves first identifying the origin of the name, and then back-transliterating the name to all possible Chinese characters using language-specific mappings. To reduce the massive number of possibilities for computation, we apply a three-tier filtering process by filtering first through a set of attested bigrams, then through a set of attested terms, and lastly through the WWW for a final validation. We illustrate the approach with English-to-Japanese back-transliteration. Against test sets of Japanese given names and surnames, we have achieved average precisions of 73% and 90%, respectively. 1 Introduction Multilingual processing in the real world often involves dealing with proper names. Translations of names, however, are often missing in bilingual resources. This absence adversely affects multilingual applications such as machine translation (MT) or cross language information retrieval (CLIR) for which names are generally good discriminating terms for high IR performance (Lin et al., 2003). For language pairs with different writing systems, such as Japanese and English, and for which simple string-copying of a name from one language to another is not a solution, researchers have studied techniques for transliteration, i.e., phonetic translation across languages. For example, European names are often transcribed in Japanese using the syllabic a katakana alphabet. Knight and Graehl (1998) used bilingual English-katakana dictionary, katakana-to-English phoneme mapping, and the a CMU Speech Pronunciation Dictionary to create a series of weighted finite-state transducers between English words and katakana that produce and rank transliteration candidates. Using similar methods, Qu et al. (2003) showed that integrating automatically discovered transliterations of unknown katakana sequences, i.e. those not included in a large Japanese-English dictionary such as EDICT1, improves CLIR results. Transliteration of names between alphabetic and syllabic scripts has also been studied for languages such as Japanese/English (Fujii & Ishikawa, 2001), English/Korean (Jeong et al., 1999), English/Arabic (Al-Onaizan and Knight, 2002). and In work closest to ours, Meng et al (2001), working in cross-language retrieval of phonetically transcribed spoken text, studied how to transliterate names into Chinese phonemes (though not into Chinese characters). Given a list of identified names, Meng et al. first separated the names into Chinese names and English names. Romanized Chinese names were detected by a left- to-right longest match segmentation method, using the Wade-Giles2 and the pinyin syllable inventories in sequence. If a name could be segmented successfully, then the name was considered a Chinese name. As …

Content truncated to 500 words