Translation & Transform Algorithm of Query Sentence in Cross-Language Information Retrieval

Translation & Transform Algorithm of Query Sentence in Cross-Language Information Retrieval∗ Xiao-fei ZHANG1 Ke-liang ZHANG1, 2 He-yan HUANG1 1 Research Center of Computer & Language Information Engineering, CAS, Beijing 100097 2 Nanjing University of Science and Technology, Nanjing 210049 {xiaofei.zhang, keliang.zhang, heyan.huang}@hjtek.com Abstract. Based on large-scale bilingual corpora and the theories of vector space model and lexical mutual information, this paper explores the application of the traditional monolingual IR technology to converting the translation of query sentence to the computation of the boost value of query keyword translations in the bilingual dictionary, so that the target language query sentence is reconstructed. The experiment finds a 92.8% precision rate in the first 10 retrieved documents and an 88.9% precision rate in the first 100 retrieved documents. Keywords: cross-language information retrieval; query sentence; translation & transform algorithm 1 Introduction Cross-language information retrieval (CLIR）can be briefly described as follows: (1) The retrieval request is posed in a given language; (2) The computer automatically searches the documents written in another language or other languages; (3) The retrieval results can be delivered, by way of automatic translation, in the language specified by the user. CLIR integrates the traditional monolingual information retrieval technology with machine translation (MT) technology. It is becoming a worldwide subject of crucial importance in the Information Age, as proved in some way by the fact that CLIR is included from time to time as an important subtask in the annual event of Text Retrieval Conference (TREC). In CLIR systems, the query sentence is usually input as a combination of a series of keywords rather than a sentence in its exact sense. Due to the absence in the series of query keywords of necessary syntactic and semantic information, traditional MT technology [1, 2] can not be readily used for the precise translation of the query sentence. In this paper, a new translation and transform algorithm of query sentence is proposed. This approach is based on large-scale corpora, traditional monolingual IR technology and the theories of vector space model (VSM) and lexical mutual information [3, 4]. 2 Query and Query Transform In cross-language information retrieval, the input is often a combination of a series of keywords rather than a complete sentence. This sequence of query keywords lacks necessary contextual and syntactico- semantic information, so they can not be translated by traditional MT technology in a direct and easy way. Neither can the translation problem be resolved by simply looking up the bilingual dictionary. For example, the English word “bank” corresponds to two Chinese meanings in a typical English-Chinese dictionary: “银行” and “河岸”. Then the problem arises: should the word “bank” be translated into “银 行” or “河岸”? In common sense, when the user inputs {bank; credit} as a query, we may well conclude that he is most likely looking for information about “银行” and “信用”, although we cannot completely exclude ∗ This research is supported by the National Science Foundation of China under Grant No. 60502048. 467 the small possibility of interpreting “bank” as …

Content truncated to 500 words