Yet Another Chart-Based Technique for Parsing Ill-Formed Input

Yet Another Chart-Based Technique for Parsing Ill-Formed Input Tsuneaki Kato NTT Information and Communication Systems Laboratories 1-2356 Take, Yokosuka-shi, Kanagawa, 238-03 JAPAN kato@nttnly.ntt.jp Abstract A new chart-based technique for parsing ill-formed input is proposed. This can process sentences with unknown/misspelled words, omitted words or extraneous words. This generalized parsing strategy is, similar to Mellish's, based on an active chart parser, and shares the many advantages of Mellish's technique. It is based on pure syntactic knowledge, it is independent of all grammars, and it does not slow down the original parsing operation if there is no ill-formedness. However, unlike Mellish's technique, it doesn't employ any complicated heuristic parameters. There are two key points. First, instead of using a unified or interleaved process for finding errors and correcting them, we separate the initial error detection stage from the other stages and adopt a version of bi-directional parsing. This effectively prunes the search space. Second, it employs normal top-down parsing, in which each parsing state reflects the global context, instead of top- down chart parsing. This enables the technique to determine the global plausibility of candidates easily, based on an admissible A* search. The proposed strategy could enumerate all possible minimal-penalty solutions in just 4 times the time taken to parse the correct sentences. 1 Introduction It is important that natural language interface systems have the capability of composing the globally most plausible explanation if a given input can not be syntactically parsed. This would be useful for handling erroneous inputs from the user and for offsetting grammar and lexicon insufficiency. Also, such a capability could be applied to deal with the ungrammatical sentences and sentence fragments that frequently appear in spoken dialogs (Bear, Dowding and Shriberg, 1992). Several efforts have been conducted to achieve this objective ((Lang, 1988; Saito and Tomita, 1988), for example.) One major decision to be made in designing this capability is whether knowledge other than purely syntactic knowledge is to be used. Other- than syntactic knowledge includes grammar specific recovery rules such as recta-rules (Weishedel and Sondheimer, 1983), semantic or pragmatic knowledge which may depend on a particular domain (Carbonell and Hayes, 1983) or the characteristics of the ill-formed utterances observed in human discourse (Hindle, 1983). Although it is obvious that the utilizing such knowledge allows us to devise more powerful strategies, we should first determine the effectiveness of using only syntactic knowledge. Moreover, the result can be applied widely, as using syntactic knowledge is a base of the most of strategies. One significant advance in the usage of syntactic knowledge was contained in the technique proposed by Mellish (1989). It can handle not only unknown/misspelled words, but also omitted words and extraneous words in sentences. It can deal with such problems, and develop plausible explanations quickly since it utilizes the full syntactic context by using an active chart parser (Kay, 1980; Gazdar and Mellish, 1989). One problem with his technique is that its performance heavily depends on how the search heuristics, which is implemented as a score …

Content truncated to 500 words