A News Analysis System

A News Analysis System Robert J. KUHNS Artificial Intelligence Center Arthur D. Little, Inc. Cambridge, MA 02140 USA Abstract This paper describes a prototype news analysis system Which classifies and indexes news stories in real time. The system extracts stories from a newswire, parses the sentences of the story, and then maps the syntactic structures into a concept base. This process results in an index containing both ger~eral categories and specific details. Central to this system is a Government-Binding parser which processes each sentence of a news item. The system is completely modular and can be interfaced with different news feeds or concept bases. 1,0 Introduction This paper reports on a prototype news analysis system (NAS) which classifies and indexes news stories in real time. That is, the system receiving reports from a newswire is capable of classifying the reports and constructing an index of them. Once a news item is classified, it can then be routed to users for whom the story is pertinent. This system, which runs as an independent background process, is automatic and greatly reduces the amount of irrelevant information users must cope with. From a theoretical view, one significant aspect of the system is that its parser is a deterministic princlple-based Government-Binding (GB) system. Basing NAS on such a processor demonstrates capabilities of a syntax-oriented natural %anguage parser and how linguistic and world knowledge (primarily financial) can be interfaced to provide a useful application. While a pre-prototype of NAS was written in ZetaLisp on Symbolics and later ported to the TI Explorer and Explorer II, the current version is implemented in Symbolics Cool,on Lisp. ~ r o u n d Much of the text processing work has focused on methods for obtaining information or retrieving texts from large databases. Approaches are wide and inc]ude key wording, statistical analysis, pattern matching, and a method using lexical, syntactic, and semantic filters. However, there are other applications for which these techniques seem inadequate. (/Hayes et al. 1988/ does describe a strictly pattern matching approach to news categorization.) For instance, consider the news and financial industries where those who gather and report news or trade stocks and bonds must read, analyze, and react to current electronic information from many different sources almost instantly. This need to process information within seconds, coupled with the fact that there is already ~t overwhelming amount of information that individuals must sort through in order to find relevant news, clearly shows the need for rapid and accurate indexing and routing systems. It was in this context that NAS was developed. The goal was to build a system with the capability of processing news stories received from active newswires, i.e., to be able to categorize each story against a set of general topics and more detailed subtopics in a matter of seconds. These stories and their associated indexes can then be routed to those interested individuals, thereby helping to reduce the load of irrelevant information that they must see in order to find the items …

Content truncated to 500 words