Everybody loves a rich cousin: An empirical study of transliteration through bridge languages

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 420–428, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Everybody loves a rich cousin: An empirical study of transliteration through bridge languages Mitesh M. Khapra Indian Institute of Technology Bombay, Powai, Mumbai 400076, India miteshk@cse.iitb.ac.in A Kumaran Microsoft Research India, Bangalore, India a.kumaran@microsoft.com Pushpak Bhattacharyya Indian Institute of Technology Bombay, Powai, Mumbai 400076, India pb@cse.iitb.ac.in Abstract Most state of the art approaches for machine transliteration are data driven and require sig- nificant parallel names corpora between lan- guages. As a result, developing translitera- tion functionality among n languages could be a resource intensive task requiring paral- lel names corpora in the order of nC2. In this paper, we explore ways of reducing this high resource requirement by leveraging the avail- able parallel data between subsets of the n lan- guages, transitively. We propose, and show empirically, that reasonable quality transliter- ation engines may be developed between two languages, X and Y , even when no direct par- allel names data exists between them, but only transitively through language Z. Such sys- tems alleviate the need for O(nC2) corpora, significantly. In addition we show that the per- formance of such transitive transliteration sys- tems is in par with direct transliteration sys- tems, in practical applications, such as CLIR systems. 1 Introduction Names and Out Of Vocabulary (OOV) terms appear very frequently in written and spoken text and hence play a very important role in several Natural Lan- guage Processing applications. Several studies have shown that handling names correctly across lan- guages can significantly improve the performance of CLIR Systems (Mandl and Womser-Hacker, 2004) and the utility of machine translation systems. The fact that translation lexicons or even statistical dic- tionaries derived from parallel data do not provide a good coverage of name and OOV translations, un- derscores the need for good transliteration engines to transform them between the language. The importance of machine transliteration, in the above context, is well realized by the research com- munity and several approaches have been proposed to solve the problem. However, most state of the art approaches are data driven and require significant parallel names corpora between languages. Such data may not always be available between every pair of languages, thereby limiting our ability to support transliteration functionality between many language pairs, and subsequently information access between languages. For example, let us consider a practi- cal scenario where we have six languages from four different language families as shown in Figure 1. The nodes in the graph represent languages and the edges indicate the availability of data between that language pair and thus the availability of a Machine Transliteration system for that pair. It is easy to see the underlying characteristics of the graph. Data is available between a language pair due to one of the following three reasons: Politically related languages: Due to the political dominance of English it is easy to obtain parallel names data …

Content truncated to 500 words