An Integrated Archictecture for Shallow and Deep Processing

An Integrated Architecture for Shallow and Deep Processing Berthold Crysmann, Anette Frank, Bernd Kiefer, Stefan M¨uller, G¨unter Neumann, Jakub Piskorski, Ulrich Sch¨afer, Melanie Siegel, Hans Uszkoreit, Feiyu Xu, Markus Becker and Hans-Ulrich Krieger DFKI GmbH Stuhlsatzenhausweg 3 Saarbr¨ucken, Germany whiteboard@dfki.de Abstract We present an architecture for the integra- tion of shallow and deep NLP components which is aimed at flexible combination of different language technologies for a range of practical current and future appli- cations. In particular, we describe the inte- gration of a high-level HPSG parsing sys- tem with different high-performance shal- low components, ranging from named en- tity recognition to chunk parsing and shal- low clause recognition. The NLP com- ponents enrich a representation of natu- ral language text with layers of new XML meta-information using a single shared data structure, called the text chart. We de- scribe details of the integration methods, and show how information extraction and language checking applications for real- world German text benefit from a deep grammatical analysis. 1 Introduction Over the last ten years or so, the trend in application- oriented natural language processing (e.g., in the area of term, information, and answer extraction) has been to argue that for many purposes, shallow natural language processing (SNLP) of texts can provide sufficient information for highly accurate and useful tasks to be carried out. Since the emer- gence of shallow techniques and the proof of their utility, the focus has been to exploit these technolo- gies to the maximum, often ignoring certain com- plex issues, e.g. those which are typically well han- dled by deep NLP systems. Up to now, deep natural language processing (DNLP) has not played a sig- nificant role in the area of industrial NLP applica- tions, since this technology often suffers from insuf- ficient robustness and throughput, when confronted with large quantities of unrestricted text. Current information extractions (IE) systems therefore do not attempt an exhaustive DNLP analy- sis of all aspects of a text, but rather try to analyse or “understand” only those text passages that contain relevant information, thereby warranting speed and robustness wrt. unrestricted NL text. What exactly counts as relevant is explicitly defined by means of highly detailed domain-specific lexical entries and/or rules, which perform the required mappings from NL utterances to corresponding domain knowl- edge. However, this “fine-tuning” wrt. a particular application appears to be the major obstacle when adapting a given shallow IE system to another do- main or when dealing with the extraction of com- plex “scenario-based” relational structures. In fact, (Appelt and Israel, 1997) have shown that the cur- rent IE technology seems to have an upper perfor- mance level of less than 60% in such cases. It seems reasonable to assume that if a more accurate analy- sis of structural linguistic relationships could be pro- vided (e.g., grammatical functions, referential rela- tionships), this barrier might be overcome. Actually, the growing market needs in the wide area of intel- ligent information management systems seem to re- quest such a break-through. In …

Content truncated to 500 words