Scruffy Text Understanding: Design and Implementation of the NOMAD System

Scruffy Text Understanding: Design and Implementation of the NOMAD System Richard H. Granger, Chris J. Staros, Gregory S. Taylor, like Yoshii Artificial Intelligence Project Department of Information and Computer Science University of California Irvine, California 92717 ABSTRACT The task of understanding unedited naval ship-co-shore messages is implemented in the presence of a large database of domain specific knowledge. The program uses internal syntactic and semantic expectations to analyze the texts and co correct errors that arise during understanding, such as syntactic errors, missing punctuation, and errors of spelling and usage. The output of the system is a veil-formed English translation of the message. This paper describes some of the knowledge mechanisms that have been implemented in the NOMAD system. I. Introduction Consider the following message, LOCKED ON OPEN FIRED DIW. This is an actual naval message containing sentence boundary problems, missing subjects and objects, an incorrect verb conjugation, and an abbreviation for "dead in water." The NAVY receives many thousands of short messages like the one above in very "scruffy" form, and these messages have =o be put into a more readable form before they can be passed through many hands. Hence there is an obvious benefit co partially automating this encoding process. Most large text-understanding systems today would not be able to automate the encoding process mentioned above because they were designed under the assumption chat the input text consists of well-formed and logical sentences such as newspaper stories and other edited texts. The NOMAD system, however, was designed to understand naval text that contains ungra~aticai or only partially complete sentences. This paper explains some knowledge mechanisms that underlie the reader's ability Co understand scruffy text and how these mechanisms are implemented within the NOMAD system. This research was supported in part by the Naval Ocean Systems Center grant N00123-81-C-I078. II. Categories of Errors We have encountered the following problems in understanding Navy messages. They are listed in the order of frequency of their occurrences. The order was determined by examining the list of messages provided by the Naval Ocean Systems Center. For each error type, NOMAD's method of recognizing and correcting the problem are described, and the ~Jodule which is responsible for the correction is identified. A. Unknown words Consider the message, PEGASUS FRD 2 TALOS AT VICTOR NOMAD does not i-~ediately recognize "FRD" as a word in the dictionary. Often the message sender viii use an ad hoc abbreviation of a word or misspell a word. Any word not found Co be in the dictionary is first put through a simple spelling correction procedure. If none of the possible corrections are recognizable then a morphological analyzer i8 applied to recognize different possible conjugations of a a verb. If this fails, a mechanism called FOUL-UP (Granger, 1977) is triggered. The FOUL-UP mechanism handles unknown words by using the program's own syntactic and semantic expectations Co create a temporary definition Chat would allow ic continue normally. FOOL-UP would later revise the definition of the unknown word by combining the …

Content truncated to 500 words