The Murasaki Project: Multilingual Natural Language Understanding

THE MURASAKI PROJECT: MULTILINGUAL NATURAL LANGUAGE UNDERSTANDING Chinatsu Aone, Hatte Blejer, Sharon Flank, Douglas McKee, Sandy Shinn Systems Research and Applications (SRA) 2000 15th Street North Arlington, VA 22201 ABSTRACT This paper describes a multilingual data extraction system under development for the Department of Defense (Do[)). The system, called Murasa.ki, processes Spanish and Japanese newspaper articles reporting AIDS disease statistics. Key to Murasaki's design is its language-independent and domain-independent architecture. The system consists of shared processing modules across the three languages it currently handles (English, Japanese, and Spanish), shared general and domain-specific knowledge bases, and separate data modules for language-specific knowledge such as grammars, lexicons, morphological data and discourse data. This data-driven architecture is crucial to the success of Murasaki as a language- independent system; extending Murasaki to additional languages can be done for the most part merely by adding new data. Some of the data can be added with user-friendly tools, others by exploiting existing on-line data or by deriving relevant data from corpora. I. INTRODUCTION Project Murasaki is a 30-month project for DoD to design and develop a data extraction prototype, operative in Spanish and Japanese and extensible to other languages. Using SRA's core natural language processing (NLP) software, SOLOMON, Project Murasaki extracts information from newspaper ar- ticles and TV transcripts in Japanese and from newspaper articles from a variety of Spanish-speaking countries. The topic of the articles and transcripts is the disease AIDS. The extracted information- some in a canonical form and some as it appears in the input texts - is stored in an object- oriented database schema implemented in a recently released multilingual version of the Sybase RDBMS. Project Murasaki has been under development since Octo- ber 1990 and will be delivered to DoD in June 1993. The goal of the project was to extend SOLOMON's data extrac- tion capabilities, hitherto used for English texts, to Spanish and Japanese. It was explicitly requested that Murasaki be as language-independent and domain-independent as possi- ble and be extensible to additional languages and domains ultimately. SOLOMON reflects six years of development. From its inception, language and domain independence have been deliberate design goals. Murasaki was our first extensive use of SOLOMON for languages other than English and thus the first testing-ground for its claimed language independence. SOL()MON had been used and continues to be used across a variety of domains over the past six years. In the MUC-4 conference, SRA demonstrated a single system extracting information about Latin American terrorism from newspaper articles in all three languages, using Spanish and Japanese data modules developed for Murasaki and terrorism vocabulary in Spanish and Japanese acquired in the two weeks prior to the demonstration (cf. [1, 2]). SOLOMON's architecture did not change significantly during the course of Murasaki. For the most part, its claim to language independence was borne out. Below, we will discuss how we have extended it to increase its language independence. 2. UNIQUE FEATURES OF MURASAKI 2.1. Modular Architecture Murasaki is composed of shared processing modules across …

Content truncated to 500 words