The Experience of Developing a Large-Scale Natural Language Text Processing System: Critique

TIlE EXPERIENCE OF DEVELOPING A LARGE-SCALE NATURAL LANGUAGE TEXT PROCFASSING SYSTEM: CRITIQUE Stephen D. Richardson and Lisa C. Braden-Harder IBM Thomas J. Watson Research Center P.O. Box 704 Yorktown Heights, New York 10598 Abstract This paper describes our experience in devel- oping the CRITIQUE system. It describes three application areas in which the system is being used and discusses some characteristics of CRITIQUE which we believe are applicable to large-scale natural language systems in general: performance, robustness, flexibility, presentation, and accuracy. Introduction CRITIQUE is a large-scale natural language text processing system that identifies grammar and style errors in English text. This advanced prototype, which is currently being developed at IBM Research, is based on a broad-coverage na- tural language parser (Richardson, 1985). The parser provides a unique approximate syntactic parse for a large percentage of English text and diagnoses over 100 grammar and style errors. Earlier writing-aid systems, such as Writer's Workbench (Macdonald, et al, 1982), contain functions which identify parts-of-speech of words, perform string-level phrase identification, and generate readability statistics. Similar func- tions are apparent in many systems now com- mercially available, some of which are described in an issue of The SeyboM Report (1984). To our knowledge, however, no other system uses a parser that produces complete structural analyses for sentences. CRITIQUE is an extension of the EPISTLE project which began in 1980 (Heidom, et al, 1982). The parser and grammar are implemented in PLNLP (the Programming Language for Na- tural language Processing), developed by George Heidom. PEG (the PLNLP English Grammar) has been written by Karen Jensen, and the style rules were written by Yael Ravin. Today, CRITIQUE is being tested in a variety of appli- cations ranging from office correspondence and technical documentation to student essays. Also, PLNLP and PEG have been incorporated into several other research applications, such as ma- chine translation systems. At the 1986 A('L meeting, Gary Hendrix de- scribed his experience in developing a natural language interface for real users (Hendrix, 1986). In contrast with user interface systems, we con- sider CRITIQUE to be a text processing system. The latter may be distinguished from the former by its broad coverage of texts that were prepared independently to communicate ideas and not strictly to interact with a computer system. Until now, the experience of developing a large-scale natural language text processing system has not been discussed in the literature. This paper first describes the overall process- ing in the CRITIQUE system. Then it describes three application areas in which the system is being used. The remaining sections discuss some characteristics of CRITIQUE which we believe are applicable to large-scale natural language systems in general: performance, robustness, flexibility, presentation, and accuracy. The dis- cussion draws on our experience in all three ap- plication areas. Processing in CRITIQUE CRITIQUE processes text in six steps. The first step determines sentence, heading, and par- agraph boundaries. In the next step, lexical processing identifies unrecognized words and awkward phrases. The on-line dictionary which is used includes more than …

Content truncated to 500 words