Robust Parsing Based on Discourse Information: Completing partial parses of ill-formed sentences on the basis of discourse information

Robust Parsing Based on Discourse Information: Completing partial parses of ill-formed sentences on the basis of discourse information Tetsuya Nasukawa IBM Research, Tokyo Research Laboratory 1623-14, Shimotsurmna, Yamato-shi, Kanagawa-ken 242, Japan nasukawaOtrl, vnet. ibm. com Abstract In a consistent text, many words and phrases are repeatedly used in more than one sentence. When an identical phrase (a set of consecutive words) is repeated in different sentences, the constituent words of those sentences tend to be associated in identical modification patterns with identi- cal parts of speech and identical modifiee- modifier relationships. Thus, when a syntactic parser cannot parse a sentence as a unified structure, parts of speech and modifiee-modifier relationships among morphologically identical words in com- plete parses of other sentences within the same text provide useful information for obtaining partial parses of the sentence. In this paper, we describe a method for completing partial parses by maintaining consistency among morphologically identi- cal words within the same text as regards their part of speech and their modifiee- modifier relationship. The experimental results obtained by using this method with technical documents offer good prospects for improving the accuracy of sentence analysis in a broad-coverage natural lan- guage processing system such as a machine translation system. 1 Introduction In order to develop a practical natural language pro- cessing (NLP) system, it is essential to deal with ill-formed sentences that cannot be parsed correctly according to the grammar rules in the system. In this paper, an "ill-formed sentence" means one that cannot be parsed as a unified structure. A syntac- tic parser with general grammar rules is often un- able to analyze not only sentences with grammati- cal errors and ellipses, but also long sentences, ow- ing to their complexity. Thus, ill-formed sentences include not only ungrammatical sentences, but also some grammatical sentences that cannot be parsed as unified structures owing to the presence of un- known words or to a lack of completeness in the syntactic parser. In texts from a restricted domain, such as computer manuals, most sentences are gram- matically correct. However, even a well-established syntactic parser usually fails to generate a unified parsed structure for about 10 to 20 percent of all the sentences in such texts, and the failure to generate a unified parsed structure in syntactic analysis leads to a failure in the output of a NLP system. Thus, it is indispensable to establish a correct analysis for such a sentence. To handle such sentences, most previous ap- proaches apply various heuristic rules (Jensen et al., 1992; Douglas and Dale, 1992; Richardson and Braden-Harder, 1988), including • Relaxing constraints in the condition part of a grammatical rule, such as number and gender constraints • Joining partial parses by using meta rules. Either way, the output reflects the general plausibil- ity of an analysis that can be obtained from infor- mation in the sentence; however, the interpretation of a sentence depends on its discourse, and incon- sistency with recovered parses that contain different analyses of the same phrase …

Content truncated to 500 words