An Open Distributed Architecture for Reuse and Integration of Heterogeneous NLP Components

An Open Distributed Architecture for Reuse and Integration of Heterogeneous NLP Components R4mi Zajac, Mark Casper and Nigel Sharpies Computing Research Laboratory New-Mexico State University {zaj ac, mc asper, nige 1}©crl. nmsu. edu Abstract The shift from Computational Linguistics to Language Engineering is indicative of new trends in NLP. This paper reviews two NLP engineering problems: reuse and integration, while relating these concerns to the larger context of applied NLP. It presents a software architecture which is geared to support the development of a va- riety of large-scale NLP applications: In- formation Retrieval, Corpus Processing, Multilingual MT, and integration of Speech Components. 1 Introduction The shift from Computational Linguistics to Lan- guage Engineering 1 is indicative of new trends in NLP. We believe that it is not simply a new fash- ion but that it is indicative of the growing matura- tion of the field, as also suggested by an emphasis on building large-scale systems, away from toy research systems. There is also an increasing awareness that real-size systems are not mere scaled-up toy systems but that they present an altogether qualitatively dif- ferent set of problems that require new tools and new ideas, as clearly exemplified by recent projects and programs such as Pangloss (Frederking et al. 94), Tipster (ARPA 94), and Verbmobil (GSrz et al. 96). Natural language engineering addresses some tra- ditional issues in software engineering: robustness, testing and evaluation, reuse, and development of large-scale applications (see e.g., (Sommerville 96) for an overview). These issues have been and are the topic of a number of NLP projects and pro- grams: TSNLP, DECIDE, Tipster, MUC, TREC, Multext, Multilex, Genelex, Eagles, etc. This paper reviews two domains of problems in natural language 1To use the name of two well-known NLP journals. engineering: reuse and integration in the context of software architectures for Natural Language Pro- cessing. The emphasis is put on reuse of NLP soft- ware, components and their integration in order to build large-scale applications. Also relevant to this presentation are topics such as integration of hetero- geneous components for building hybrid systems or for integrating speech and other "higher-level" NLP components (section 2). Section 3 presents the Corelli Document Process- ing Architecture, a new software architecture for NLP which is designed to support the development of a variety of large- scale NLP applications: Infor- mation Retrieval, Corpus Processing, Multilingual MT, and integration of Speech with other NLP com- ponents. 2 Reuse in NLP There is an increasing amount of shared corpora and lexical resources that are being made available for NLP researchers through managed data repositories such as LDC, CLR, ELRA, etc. (see e.g., (Wilks et al. 92) for an overview of these repositories). These resources constitute the basic raw materials for building NLP software but not all of these re- sources can be readily used: they might be available in formats that require extensive pre-processing to transform them into resources that are tractable by NLP software. This pre-processing cannot usually be fully automated and is therefore …

Content truncated to 500 words