Morphological Processing in the Nabu System

MORPHOLOGICAL PROCESSING IN THE NABU SYSTEM Jonathan Slocum Microelectronics and Computer Technology Corporation (MCC) 3500 West Balcones Center Drive Austin, Texas 78759 ABSTRACT The Nabu morphological processor is designed to perform a number of different func- tions, of which five have so far been identified: analysis, guessing (about unknown words), syn- thesis, defaulting (proposing the most likely in- flectional paradigm for a new base form), and coding (producing all possible inflectional paradigm variants for a new base form). Com- plete or very substantial analyzers have been produced for a number of diverse languages; other functions have been implemented as well. This paper discusses our design philosophy, as well as our technique and its implementation. INTRODUCTION Nabu is a multilingual Natural Language Processing system under development in the Human Interface Laboratory at MCC, for shareholder companies. Its morphological com- ponent is designed to perform a number of dif- ferent functions. This has been used to produce a complete analyzer for Arabic; very substantial analyzers for English, French, German, and Spanish; and small collections of rules for Rus- sian and Japanese. In addition, other functions have been implemented for several of these lan- guages. In this paper we discuss our philosophy, which constrained our design decisions; elaborate some specific functions a morphological com- ponent should support; survey some competing approaches; describe our technique, which provides the necessary functionality while meet- ing the other design constraints; and support our approach by characterizing our success in developing/testing processors for various com- binations of language and function. DESIGN PHILOSOPHY Before we set about designing our mor- phological processor, we elaborated our philosophical commitments regarding an NLP system. These include: (1) multilingual applica- tion, (2) fault-tolerant, fail-soft behavior, (3) rule reversibility, (4) disparate functionality, (5) inherent parallelism, (6) grammatical clarity, and (7) rigorous testing. MULTILINGUAL APPLICATION The algorithms and their software instantia- tion must admit application to any human lan- guage likely to confront our system; these in- clude the languages of major demographic and/or economic significance (and their relatives). Our selected representatives are English, French, German, Spanish, Russian, Arabic, Chinese, and Japanese. FAULT-TOLERANT, FAIL-SOFT BEHAVIOR Real-world NLP applications, whether for text or interactive dialog, will be confronted with numerous errors of various types. As far as users are concerned, guaranteed failure in the presence of any error is intolerable: a system must overcome simple mistakes without discern- able problems. For example, insignificant typing and/or spelling mistakes should be ignored, as should minor morphological blunders. Users do not like to be asked for corrections of (seemingly) simple mistakes, and of course printed texts cannot be queried in any case. In the presence of more serious problems, perfor- mance should degrade only gradually. This is nothing more than a commitment to understanding the utterance, rather than punishing the user for errors in it. We contend that human-like fault-tolerant, fail-soft behavior must be incorporated in the fundamental design of a system: it cannot be tacked-on after system development. Creating an applied system for a "perfect" natural language that …

Content truncated to 500 words