Automatic Suggestion of Significant Terms for a Predefined Topic

I Automatic Suggestion of Significant Terms for a Predefined Topic Joe Zhou and Pe~Dapkus LEXIS-NEXIS, a Division of Reed Elsevier, Inc. 9555 Springboro Pike Miamisburg, OH 45342 {joez,peted} @ lexis-nexis.com ABSTRACT This paper presents a preliminary experiment in automatically suggesting significant terms for a predefined topic. The general method is to compare a topically focused sample created around the predefined topic with a larger and more general base sample. A set of statistical measures are used to identify significant word units in both samples. Identification of single word terms is based on the notion of word intervals. Two-word terms are identified through the computation of mutual information, and the extension of mutual information assists in capturing multi-word terms. Once significant terms of all these three types are identified, a comparison algorithm is applied to differentiate terms across the two data samples. If significant changes in the values of certain statistical variables are detected, associated terms will selected as being topic-oriented and included in a suggested list. To check the quality of the suggested terms, we compare them against terms manually determined by the domain expert. Though overlaps vary, we find that the automatical suggestion provides more terms that are useful for describing the predefined topic. 1. INTRODUCTION As we are facing the growing amount of on-line text, the use of text analysis techniques to access information from electronic sources has become more popular and, at the same time, more difficult. Currently, the effectiveness of such techniques is evaluated not only on how easily they can be applied to text sources to extract information and represent it in a systematic format (Walker 1983), but also on whether they can be applied to large text corpora of several tens of thousand of words. One of the applications of text analysis is to identify and extract significant terminology from running text. Choueka (1988), for example, describes an experiment for locating interesting collocational expressions from large textual databases. A collocational expression, as Choueka defines it, is =sequences of words whose unambiguous meaning cannot be dedved from that of their components". Other representative collocation research can be found in Church and Hanks (1990) and Smadja (1993). Though all statistically-based, their definitions of collocations are different from one another. Unlike Choueka (1988), Church and Hanks (1990) identify as collocations both interrupted and uninterrupted sequences of words. Unlike Church and Hanks (1990), Smadja (1993) goes beyond the "two-word" limitation and deals with "collocations of arbitrary length". 131 The primary goal of collocation research is to build a comprehensive lexicographic toolkit, or to assist automatic language generation applications. Therefore, the focus is on the extraction of all Interesting word pattems without distinction of domain specificity. Identifying domain-specific terminology is another research effort. Gierl and Frost (1992) descdbe their approach to extracting terminological knowledge from medical texts. Following Church and Hanks (1990), they use mutual information to select significant two-word patterns, but, at the same time, a lexical inductive process is incorporated which, as they claim, can improve the …

Content truncated to 500 words