PANGLYZER: Spanish Language Analysis System

PANGLYZER: SPANISH LANGUAGE ANALYSIS SYSTEM David Farwell Steve Helmreich Wanying Jin Mark Casper Jim Hargrave Hugo Molina-Salgado Fuliang Weng Computing Research Laboratory New Mexico State University ABSTRACT The purpose of this paper is to describe the functions and procedures of the eight components of the Panglyzer Spanish analyzer of the knowledge-based engine of the Pangloss machine translation system. The Panglyzer follows a multi-pass approach consisting of preprocessing, part-of-speech tagging, phrase recognition, proper name classification, phrase analysis, clause recognition, clause analysis and reading ranking. Introduction The function of the Spanish analysis component of the PANGLOSS system, or PANGLYZER, is to provide for each clause in the input text a set of possible meaning representations ranked on the basis of likelihood. These are then handed to an augmentor which either interactively or automatically selects a representation from among the set of possibilities and fills in various sorts of information to produce a reading that is compatible with the context and that can act as a basis for generation. The approach has been to develop the system in a bottom up manner focusing on providing layer after layer of increasingly abstract analysis in a multi-pass process. This multi-pass architecture allows for semi-independent module construction, incremental development, and is amenable to robust perfor- mance. Each level of analysis is based on a focussed type of knowledge and, to the extent possible, exploits proven techniques. The levels alternate between recognition (segmentation) modules and anal- ysis modules. For example, the preprocessor segments text into words while the part-of-speech tagger analyzes their part of speech. Since a high premium has been placed on robustness, we are following an iterative approach to design which relies on rapidly producing an initial prototype and then following a short test and revise cycle. Thus, at this point, all but the deepest level of analysis produces through- put and the on-going objective is to improve the accuracy of that throughput from one test cycle to the next System Architecture The architecture of the PANGLYZER is a by-product of the bottom up approach to development described above. It consists of eight components that sequentially process the data. The first is a Pre- processor which converts a text length input ASCII character string into a file of PROLOG data struc- tures and then builds PROLOG strings corresponding to the sentences of the input text. The second is a Spanish Part-of-Speech Tagger which provides a standard data structure for each item of an input 56 PROLOG string which includes a part of speech, relevant inflectional information, and position index. The third component is the Phrase Recognizer which groups contiguous elements of the input part-of- speech tagged string into phrase length chunks, inserting brackets around the chunks and assigning phrasal categories. The fourth component is a Proper-Noun Classifier which, operating over the entire text, assigns a semantic category such as personal name, place name, company name, and so on, along with relevant inflectional information to each element of the input tagged as a proper noun. The …

Content truncated to 500 words