Implicit Ambiguity Resolution Using Incremental Clustering in Korean-to-English Cross-Language Information Retrieval

Implicit Ambiguity Resolution Using Incremental Clustering in Korean-to-English Cross-Language Information Retrieval Kyung-Soon Lee1, Kyo Kageura1, Key-Sun Choi2 1 NII (National Institute of Informatics) 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, 101-8430, Japan {kslee, kyo}@nii.ac.jp 2 Division of Computer Science, KAIST 373-1 Kusung Yusong Daejeon, 305-701, Korea kschoi@cs.kaist.ac.kr Abstract This paper presents a method to implicitly resolve ambiguities using dynamic incremental clustering in Korean-to-English cross-language information retrieval. In the framework we propose, a query in Korean is first translated into English by looking up Korean-English dictionary, then documents are retrieved based on the vector space retrieval for the translated query terms. For the top-ranked retrieved documents, query-oriented document clusters are incrementally created and the weight of each retrieved document is re-calculated by using clusters. In experiment on TREC-6 CLIR test collection, our method achieved 28.29% performance improvement for translated queries without ambiguity resolution for queries. This corresponds to 97.27% of the monolingual performance for original queries. When we combine our method with query ambiguity resolution, our method even outperforms the monolingual retrieval. 1 Introduction This paper describes a method of applying dynamic incremental clustering to the implicit resolution of query ambiguities in Korean-to-English cross-language information retrieval. The method uses the clusters of retrieved documents as a context for re-weighting each retrieved document and for re-ranking the retrieved documents. Cross-language information retrieval (CLIR) enables users to retrieve documents written in a language different from a query language. The methods used in CLIR fall into two categories: statistical approaches and translation approaches. Statistical methods establish cross-lingual associations without language translation (Dumais et al, 1997; Rehder et al, 1997; Yang et al, 1998). They require large-scale bilingual corpora. In translation approach, either queries or documents are translated. Though document translation is possible when high quality machine translation systems are available (Kwon et al, 1997; Oard and Hackett, 1997), it is not very practical. Query translation methods (Hull and Grefenstette, 1996; Davis, 1996; Eichmann et al, 1998; Yang et al, 1998; Jang et al, 1999; Chun, 2000) based on bilingual dictionaries, multilingual ontology or thesaurus are much more practical. Many researches adopt dictionary-based query translation because it is simpler and practical, given the wide availability of bilingual or multilingual dictionaries. In order to achieve a high performance CLIR using dictionary-based query translation, however, it is necessary to solve the problem of increased ambiguities of query terms. One way of resolving query ambiguities is to use the statistics, such as mutual information (Church and Hanks, 1990), to measure associations of query terms, on the basis of existing corpora (Jang et al, 1999). Document clusters, widely adopted in various applications such as browsing and viewing of document results (Hearst and Pedersen, 1996) or topic detection (Allan et al, 1998), also reflect the association of terms and documents. Lee et al (2001) showed that incorporating a document re-ranking method based on document clusters into the vector space retrieval achieved the significant improvement in monolingual IR, as it contributed to resolving ambiguities caused by polysemous query terms. The noise or ambiguity produced …

Content truncated to 500 words