Building a Large Thesaurus for Information Retrieval

BUILDING A LARGE THESAURUS FOR INFORMATION RETRIEVAL Edward A. Fox** and J. Terry Nutter Department of Computer Science Virginia Tech, Blacksburg VA 24061 Thomas Ahlswede and Martha Evens Computer Science Department Illinois Institute of Technology, Chicago IL 60616 Judith Markowitz Navistar International Oakbrook Terrace, IL 60181 ABSTRACT Information retrieval systems that support searching of large textual databases are typically accessed by trained search intermediaries who provide assistance to end users in bridging the gap between the languages of authors and inquirers. We are building a thesaurus in the form of a large semantic network .to support interactive query expan- sion and search by end users. Our lexicon is being built by analyzing and merging data from several large English dic- tionaries; testing of its value for rea'ieval is with the SMART and CODER systems. 1. Introduction Though computer systems aiding retrieval from bibliographic or full-text databases have been available for more than two decades, it is only in recent years that many people are becoming concerned about the serious limi- tations of those systems regarding effectiveness in finding desired references (Blair and Maron 1985). Like others, though, we are convinced that easy-to-apply automatic methods can help solve this problem (see argument in Salton 1986). Indeed, automatic approaches seem essential since many end-users want to search without involving trained intermediaries (Ojala 1986). However, since the fundamental issue is one of mismatch between language use of document authors and inquirers, leading to uncertainties regarding whether a particular item should be retrieved (Chen and Dhar 1987), we are also convinced that computational linguistics is essential for a complete solution. Since many queries are simply sets of lexemes or phrases, and all queries can be reduced to that form, we believe that focusing on lexical and phrasal issues may be the most appropriate strategy for applying computational linguistics to information retrieval. While good results have been achieved in applying automatic procedures to find lexically related words based on local context in a * This material is based in part upon work supported by the National Science Foundation under Grant No's. IST- 8418877, IST-8510069, and IRI-8703580; by the Virginia Center for Innovative Technology under Grant No. INF- 85-016; and by AT&T equipment contributions. **All correspondence regarding this paper should be addressed to the first author. particular collection (Attar and Fraenkel 1977), no reliable techniques exist for using collection statistics to build a thesaurus automatically, and manual construction of thesauri is both problematic and expensive (Svenonius 1986). Efforts to represent a significant portion of common sense knowledge will take years (Lenat et al. 1986), but developing knowledge resources to aid text processing is now feasible (Walker 1985). Encouraged by activities in identifying lexical and semantic information in other languages (especially since that reported in Apresyan et al. 1970), we decided to build a large, comprehensive lexicon for English that would include lexical and semantic relations (see comments below and survey in Evens et al. 1980) and be of use for information retrieval. Clearly this might also …

Content truncated to 500 words