Document Re-ranking Based on Automatically Acquired Key Terms in Chinese Information Retrieval

Document Re-ranking Based on Automatically Acquired Key Terms in Chinese Information Retrieval Yang Lingpeng, Ji Donghong, Tang Li Institute for Infocomm Research 21, Heng Mui Keng Terrace Singapore, 119613 {lpyang,dhji,tangli}@i2r.a-star.edu.sg Abstract For Information Retrieval, users are more concerned about the precision of top ranking documents in most practical situations. In this paper, we propose a method to improve the precision of top N ranking documents by reordering the retrieved documents from the initial retrieval. To reorder documents, we first automatically extract Global Key Terms from document set, then use extracted Global Key Terms to identify Local Key Terms in a single document or query topic, finally we make use of Local Key Terms in query and documents to reorder the initial ranking documents. The experiment with NTCIR3 CLIR dataset shows that an average 10%-11% improvement and 2%-5% improvement in precision can be achieved at top 10 and 100 ranking documents level respectively. 1 Introduction Information retrieval (IR) is used to retrieve relevant documents from a large document set for a given query where the query is a simple description by natural language. In most practical situations, users concern more on the precision of top ranking documents than recall because users want to acquire relevant information from the top ranking documents. Traditionally, IR system uses a one-stage or a two-stage mechanism to retrieve relevant documents from document set. For one stage mechanism, IR system only does an initial retrieval. For two-stage mechanism, besides the initial retrieval, IR system will make use of the initial ranking documents to automatically do query expansion to form a new query and then use the new query to retrieve again to get the final ranking documents. The effectiveness of query expansion mainly depends on the precision of top N (N<50) ranking documents in initial retrieval because almost all proposed automatic query expansion algorithms make use of the information in the top N retrieved. Figure 1 demonstrates the general processes of a two-stage IR system. In this paper, we propose a method to improve the precision of top N ranking documents by reordering the initially retrieved documents in the initial retrieval. To reorder documents, we first automatically extract Global Key Terms from the document set, then use the extracted Global Key Terms to identify Local Key Terms in a single document or query topic, finally we make use of the Local Key Terms in queries and documents to reorder the initial ranking documents. Although our method is general and can apply to any languages, in this paper we’ll only focus on the research on Chinese IR system. F i g . 1 T r a d i t i o n a l P r o c e s s o f t w o - s t a g e s I R O r i g i n a l Q u e r y E x p a n d e d Q u e r y I n i t i a l …

Content truncated to 500 words