Learning Patterns from the Web to Translate Named Entities for Cross Language Information Retrieval

Learning Patterns from the Web to Translate Named Entities for Cross Language Information Retrieval Yu-Chun Wang†‡ Richard Tzong-Han Tsai§∗ Wen-Lian Hsu† †Institute of Information Science, Academia Sinica, Taiwan ‡Department of Eletrical Engineering, National Taiwan University, Taiwan §Department of Computer Science and Engineering, Yuan Ze University, Taiwan albyu@iis.sinica.edu.tw thtsai@saturn.yzu.edu.tw hsu@iis.sinica.edu.tw ∗corresponding author Abstract Named entity (NE) translation plays an important role in many applications. In this paper, we focus on translating NEs from Korean to Chinese to improve Korean-Chinese cross-language informa- tion retrieval (KCIR). The ideographic nature of Chinese makes NE translation difficult because one syllable may map to several Chinese characters. We propose a hybrid NE translation system. First, we integrate two online databases to ex- tend the coverage of our bilingual dic- tionaries. We use Wikipedia as a trans- lation tool based on the inter-language links between the Korean edition and the Chinese or English editions. We also use Naver.com’s people search en- gine to find a query name’s Chinese or English translation. The second compo- nent is able to learn Korean-Chinese (K- C), Korean-English (K-E), and English- Chinese (E-C) translation patterns from the web. These patterns can be used to extract K-C, K-E and E-C pairs from Google snippets. We found KCIR per- formance using this hybrid configura- tion over five times better than that a dictionary-based configuration using only Naver people search. Mean average precision was as high as 0.3385 and recall reached 0.7578. Our method can han- dle Chinese, Japanese, Korean, and non- CJK NE translation and improve perfor- mance of KCIR substantially. 1 Introduction Named entity (NE) translation plays an impor- tant role in machine translation, information re- trieval, and question answering. It is a chal- lenging task because, although there are many online bilingual dictionaries, they usually lack domain specific words or NEs. Furthermore, new NEs are generated everyday, but bilingual dictionaries cannot update their contents fre- quently. Therefore, it is necessary to construct a named entity translation (NET) system. Economic ties between China and Korea have become closer as China has opened its mar- kets further, and demand for the latest news and information from China continues to grow rapidly in Korea. One key way to meet this demand is to retrieve information written in Chinese by using Korean queries, referred to as Korean-Chinese cross-language information retrieval (KCIR). The main challenge involves translating NEs because they are usually the main concepts of queries. In (Chen et al., 1998), the authors romanized Chinese NEs and selected their English transliterations from English NEs extracted from the Web by comparing their phonetic similarities with Chinese NEs. Yaser Al-Onaizan (Al-Onaizan and Knight, 2002) 281 transliterated an NE in Arabic into several can- didates in English and ranked the candidates by comparing their counts in several English cor- pora. Unlike the above works, whose target lan- guages are alphabetic, in K-C translation, the target language is Chinese, which uses an ideo- graphic writing system. Korean-Chinese NET is much more difficult than NET considered in previous works because, in …

Content truncated to 500 words