Software Infrastructure for Natural Language Processing

Software Infrastructure for Natural Language Processing Hamish Cunningham Dept. Computer Science University of Sheffield 211 Portobello St. Sheffield S10 4DP hamish~dcs, shef. ac. uk Kevin Humphreys Dept. Computer Science University of Sheffield 211 Portobello St. Sheffield S10 4DP kvhOdcs, shef. ac. uk Robert Gaizauskas Dept. Computer Science University of Sheffield 211 Portobello St. Sheffield S10 4DP robertg@dcs, shef. ac. uk Yorick Wilks Dept. Computer Science University of Sheffield 211 Portobello St. Sheffield S10 4DP yorick~dcs, shef. ac. uk Abstract We classify and review current approaches to software infrastructure for research, de- velopment and delivery of NLP systems. The task is motivated by a discussion of current trends in the field of NLP and Lan- guage Engineering. We describe a sys- tem called GATE (a General Architecture for Text Engineering) that provides a soft- ware infrastructure on top of which het- erogeneous NLP processing modules may be evaluated and refined individually, or may be combined into larger application systems. GATE aims to support both re- searchers and developers working on com- ponent technologies (e.g. parsing, tagging, morphological analysis) and those work- ing on developing end-user applications (e.g. information extraction, text sum- marisation, document generation, machine translation, and second language learning). GATE promotes reuse of component tech- nology, permits specialisation and collab- oration in large-scale projects, and allows for the comparison and evaluation of al- ternative technologies. The first release of GATE is now available. 1 Introduction This paper reviews the currently available design strategies for software infrastructure for NLP and presents an implementation of a system called GATE - a General Architecture for Text Engineer- ing. By software infrastructure we mean what has been variously referred to in the literature as: soft- ware architecture; software support tools; language engineering platforms; development enviromnents. Our gloss on these terms is: common models for the representation, storage and exchange of data in and between processing modules in NLP systems, along with graphical interface tools for the management of data and processing and the visualisation of data. NLP systems produce information about texts 1, and existing systems that aim to provide software in- frastructure for NLP can be classified as belonging to one of three types according to the way in which they treat this information: additive, or markup-based: information produced is added to the text in the form of markup, e.g. in SGML (Thompson and McKelvie, 1996); referential, or annotation-based: information is stored separately with references back to the original text, e.g. in the TIPSTER architecture (Grishman, 1996); abstraction-based: the original text is preserved in processing only as parts of an integrated data structure that represents information about the text in a uniform theoretically-motivated model, e.g. attribute-value structures in the ALEP system (Simkins, 1994). A fourth category might be added to cater for those systems that provide communication and control infrastructure without addressing the text-specific needs of NLP (e.g. Verbmobil's ICE architecture (Amtrup, 1995)). We begin by reviewing examples of the three ap- proaches we sketched above (and a system that falls into the …

Content truncated to 500 words