Learning Transliteration Lexicons from the Web

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 1129–1136, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Learning Transliteration Lexicons from the Web Jin-Shea Kuo1, 2 1Chung-Hwa Telecom. Laboratories, Taiwan jskuo@cht.com.tw Haizhou Li Institute for Infocomm Research, Singapore hzli@ieee.org Ying-Kuei Yang2 2National Taiwan University of Science and Technology, Taiwan ykyang@mouse.ee. ntust.edu.tw Abstract This paper presents an adaptive learning framework for Phonetic Similarity Modeling (PSM) that supports the automatic construction of transliteration lexicons. The learning algorithm starts with minimum prior knowledge about machine transliteration, and acquires knowledge iteratively from the Web. We study the active learning and the unsupervised learning strategies that minimize human supervision in terms of data labeling. The learning process refines the PSM and constructs a transliteration lexicon at the same time. We evaluate the proposed PSM and its learning algorithm through a series of systematic experiments, which show that the proposed framework is reliably effective on two independent databases. 1 Introduction In applications such as cross-lingual information retrieval (CLIR) and machine translation (MT), there is an increasing need to translate out-of- vocabulary (OOV) words, for example from an alphabetical language to Chinese. Foreign proper names constitute a good portion of OOV words, which are translated into Chinese through transliteration. Transliteration is a process of translating a foreign word into a native language by preserving its pronunciation in the original language, otherwise known as translation-by- sound. MT and CLIR systems rely heavily on bilingual lexicons, which are typically compiled manually. However, in view of the current information explosion, it is labor intensive, if not impossible, to compile a complete proper nouns lexicon. The Web is growing at a fast pace and is providing a live information source that is rich in transliterations. This paper presents a novel solution for automatically constructing an English-Chinese transliteration lexicon from the Web. Research on automatic transliteration has reported promising results for regular transliteration (Wan and Verspoor, 1998; Li et al, 2004), where transliterations follow rigid guidelines. However, in Web publishing, translators in different countries and regions may not observe common guidelines. They often skew the transliterations in different ways to create special meanings to the sound equivalents, resulting in casual transliterations. In this case, the common generative models (Li et al, 2004) fail to predict the transliteration most of the time. For example, “Coca Cola” is transliterated into “ 可 口 可 樂 /Ke-Kou-Ke-Le/” as a sound equivalent in Chinese, which literately means “happiness in the mouth”. In this paper, we are interested in constructing lexicons that cover both regular and casual transliterations. When a new English word is first introduced, many transliterations are invented. Most of them are casual transliterations because a regular transliteration typically does not have many variations. After a while, the transliterations converge into one or two popular ones. For example, “Taxi” becomes “ 的 士 /Di-Shi/” in China and “ 德 士 /De-Shi/” in Singapore. Therefore, the adequacy of a transliteration entry could be judged by its popularity …

Content truncated to 500 words