Cross-Language Information Retrieval for Technical Documents

Cross-Language Information Retrieval for Technical Documents Atsushi Fujii and Tetsuya Ishikawa University of Library and Information Science 1-2 Kasuga Tsukuba 305-8550, JAPAN {fuj ii, ishikawa}@ulis, ac. jp Abstract This paper proposes a Japanese/English cross- language information retrieval (CLIR) system targeting technical documents. Our system first translates a given query containing tech- nical terms into the target language, and then retrieves documents relevant to the translated query. The translation of technical terms is still problematic in that technical terms are often compound words, and thus new terms can be progressively created simply by combining ex- isting base words. In addition, Japanese of- ten represents loanwords based on its phono- gram. Consequently, existing dictionaries find it difficult to achieve sufficient coverage. To counter the first problem, we use a compound word translation method, which uses a bilin- gual dictionary for base words and collocational statistics to resolve translation ambiguity. For the second problem, we propose a translitera- tion method, which identifies phonetic equiva- lents in the target language. We also show the effectiveness of our system using a test collec- tion for CLIR. 1 Introduction Cross-language information retrieval (CLIR), where the user presents queries in one language to retrieve documents in another language, has recently been one of the major topics within the information retrieval community. One strong motivation for CLIR is the growing number of documents in various languages accessible via the Internet. Since queries and documents are in different languages, CLIR requires a trans- lation phase along with the usual monolingual retrieval phase. For this purpose, existing CLIR systems adopt various techniques explored in natural language processing (NLP) research. In brief, bilingual dictionaries, corpora, thesauri and machine translation (MT) systems are used to translate queries or/and documents. In this paper, we propose a Japanese/English CLIR system for technical documents, focus- ing on translation of technical terms. Our purpose also includes integration of different components within one framework. Our re- search is partly motivated by the "NACSIS" test collection for IR systems (Kando et al., 1998) 1 , which consists of Japanese queries and Japanese/English abstracts extracted from technical papers (we will elaborate on the NAC- SIS collection in Section 4). Using this col- lection, we investigate the effectiveness of each component as well as the overall performance of the system. As with MT systems, existing CLIR systems still find it difficult to translate technical terms and proper nouns, which are often unlisted in general dictionaries. Since most CLIR systems target newspaper articles, which are comprised mainly of general words, the problem related to unlisted words has been less explored than other CLIR subtopics (such as resolution of transla- tion ambiguity). However, Pirkola (1998), for example, used a subset of the TREC collection related to health topics, and showed that com- bination of general and domain specific (i.e., medical) dictionaries improves the CLIR perfor- mance obtained with only a general dictionary. This result shows the potential contribution of technical term translation to CLIR. At the same time, note that even domain …

Content truncated to 500 words