Extending a Broad-Coverage Parser for a General NLP Toolkit

Extending A Broad-Coverage Parser for a General NLP Toolkit Hassan Alam, Hua Cheng, Rachmat Hartono, Aman Kumar, Paul Llido, Crystal Nakatsu, Fuad Rahman, Yuliya Tarnikova, Timotius Tjahjadi and Che Wilcox BCL Technologies Inc. Santa Clara, CA 95050 U.S.A. fuad@bcltechnologies.com Abstract With the rapid growth of real world applications for NLP systems, there is a genuine demand for a general toolkit from which programmers with no linguistic knowledge can build specific NLP systems. Such a toolkit should have a parser that is general enough to be used across domains, and yet accurate enough for each specific application. In this paper, we describe a parser that extends a broad-coverage parser, Minipar (Lin, 2001), with an adaptable shallow parser so as to achieve both generality and accuracy in handling domain specific NL problems. We test this parser on our corpus and the results show that the accuracy is significantly higher than a system that uses Minipar alone. 1 Introduction With the improvement of natural language processing (NLP) techniques, domains for NLP systems, especially those handling speech input, are rapidly growing. However, most computer programmers do not have enough linguistic knowledge to develop NLP systems. There is a genuine demand for a general toolkit from which programmers with no linguistic knowledge can rapidly build NLP systems that handle domain specific problems more accurately (Alam, 2000). The toolkit will allow programmers to generate natural language front ends for new and existing applications using, for example, a program-through- example method. In this methodology, the programmer will specify a set of sample input sentences or a domain corpus for each task. The toolkit will then organize the sentences by similarity and generate a large set of syntactic variations of a given sentence. It will also generate the code that takes a user’s natural language request and executes a command on an application. Currently this is an active research area, and the Advanced Technology Program (ATP) of the National Institute of Standards and Technology (NIST) is funding part of the work. In order to handle natural language input, an NLP toolkit must have a parser that maps a sentence string to a syntactic structure. The parser must be both general and accurate. It has to be general because programmers from different domains will use the toolkit to generate their specific parsers. It has to be accurate because the toolkit targets commercial domains, which usually require high accuracy. The accuracy of the parser directly affects the accuracy of the generated NL interface. In the program-through-example approach, the toolkit should convert the example sentences into semantic representations so as to capture their meanings. In a real world application, this process will involve a large quantity of data. If the programmers have to check each syntactic or semantic form by hand in order to decide if the corresponding sentence is parsed correctly, they are likely to be overwhelmed by the workload imposed by the large number of sentences, not to mention that they do not have the necessary linguistic knowledge to …

Content truncated to 500 words