Query Processing for Retrieval From Large Text Bases

QUERY PROCESSING FOR RETRIEVAL FROM LARGE TEXT BASES John Broglio and W. Bruce Croft Computer Science Department University of Massachusetts Amherst, MA 01003 ABSTRACT Natural language experiments in information retrieval have often been inconclusive due to the lack of large text bases with associated queries and relevance judgments. This pa- per describes experiments in incremental query processing and indexing with the INQUERY information retrieval sys- tem on the TIPSTER queries and document collection. The results measure the value of processing tailored for different query styles, use of syntactic tags to produce search phrases, recognition and application of generic concepts, and auto- matic concept extraction based on interword associations in a large text base. 1. INTRODUCTION: TIPSTER AND INQUERY Previous research has suggested that retrieval effective- ness might be enhanced by the use of multiple repre- sentations and by automated language processing tech- niques. Techniques include automatic or interactive in- troduction of synonyms [Har88], forms-based interfaces [CD90], automatic recognition of phrases [CTLgl], and relevance feedback [SB90]. The recent development of the TIPSTER corpus with associated queries and rel- evance judgments has provided new opportunities for judging the effectiveness of these techniques on large het- erogenous document collections. 1.1. TIPSTER Text Base and Query Topics The TIPSTER documents comprise two volumes of text, of approximately one gigabyte each, from sources such as newspaper and magazine articles and government pub- lications (Federal Register). Accompanying the collec- tions are two sets of fifty topics. Each topic is a full text description, in a specific format, of an information need. (Figure 1). Each TIPSTER topic offers several representations of the same information need. The Topic and Descrip- tion fields are similar to what might be entered as a query in a traditional information retrieval system. The Narrative field expands on the information need, giving an overview of the classes of documents which would or <top> <dom> Domain: International Economics <Title> Topic: Satellite Launch Contracts <desc> Description: Document will cite the signing of a contract or prelimi- nary agreement, or the making of a tentative reservation, to launch a commercial satellite. <narr> Narrative: A relevant document will mention the signing of a con- tract or preliminary agreement, or the making of a ten- tative reservation, to launch a commerciM satellite. <con> Concept(s): 1. contract, agreement 2. launch vehicle, rocket, payload, satellite 3. launch services, commercial space industry, commer- cial launch industry 4. Arianespace, Martin Marietta, General Dynamics, McDonnell Douglas 5. Titan, Delta II, Atlas, Ariane, Proton </top> Figure 1: A TIPSTER topic. would not be considered satisfactory, and describes facts that must be present in relevant documents, for exam- ple, the location of the company. The Concepts field lists words and phrases which are pertinent to the query. The Factors field lists constraints on the geographic and/or time frame of the query. All of these fields offer opportu- nities for different kinds of natural language processing. 1.2. The INQUERY Information Re- trieval System INQUERY is a probabilistic information retrieval sys- tem based upon a Bayesian inference …

Content truncated to 500 words