A Transitive Model for Extracting Translation Equivalents of Web Queries through Anchor Text Mining

A Transitive Model for Extracting Translation Equivalents of Web Queries through Anchor Text Mining Wen-Hsiang Lu Institute of Information Science Academia Sinica; Dept. of Computer Science and Information Engineering National Chiao Tung University Hsinchu 300, Taiwan, ROC whlu@iis.sinica.edu.tw Lee-Feng Chien Institute of Information Science, Academia Sinica Nangang 115, Taiwan, ROC lfchien@iis.sinica.edu.tw Hsi-Jian Lee Dept. of Computer Science and Information Engineering National Chiao Tung University Hsinchu 300, Taiwan, ROC hjlee@csie.nctu.edu.tw Abstract One of the existing difficulties of cross-language information retrieval (CLIR) and Web search is the lack of appropriate translations of new terminology and proper names. Different from conventional approaches, in our previous research we developed an approach for exploiting Web anchor texts as live bilingual corpora and reducing the existing difficulties of query term translation. Although Web anchor texts, undoubtedly, are very valuable multilingual and wide-scoped hypertext resources, not every particular pair of languages contains sufficient anchor texts in the Web to extract corresponding translations in the language pair. For more generalized applications, in this paper we extend our previous approach by adding a phase of transitive (indirect) translation via an intermediate (third) language, and propose a transitive model to further exploit anchor-text mining in term translation extraction applications. Preliminary experimental results show that many query translations which cannot be obtained using the previous approach can be extracted with the improved approach. 1. Introduction Cross-language information retrieval (CLIR), addressing the special need where users can query in one language and retrieve relevant documents written or indexed in another language, has become an important issue in the research of information retrieval (Dumais et al., 1996; Davis et al., 1997; Ballesteros & Croft, 1998; Nie et al., 1999). However, its application to practical Web search services has not lived up to expectations, since they suffer a major bottleneck that lacks up-to-date bilingual lexicons containing the translation of popular query terms 1 such as proper nouns (Kwok, 2001). To enable capability of CLIR, existing IR systems mostly rely on bilingual dictionaries for cross-lingual retrieval. In these systems, queries submitted in a source language normally have to be translated into a target language by means of simple dictionary lookup. These dictionary-based techniques are limited in real-world applications, since the queries given by users often contain proper nouns. Another kind of popular approaches to dealing with query translation based on corpus-based techniques uses a parallel corpus containing aligned sentences whose translation pairs are corresponding to each other (Brown et al., 1993; Dagan et al., 1993; Smadja et al., 1996). Although more reliable translation equivalents can be extracted by these techniques, the unavailability of large enough parallel corpora for various subject domains and multiple languages is still in a thorny situation. On the other hand, the alternative approach using comparable or unrelated text corpora were studied by Rapp (1999) and Fung et al. (1998). This task is more difficult due to lack of parallel correlation between document or sentence pairs. 1 In our collected query logs, most of user queries contain only one or two words, …

Content truncated to 500 words