Lenient Default Unification for Robust Processing within Unification Based Grammar Formalisms

Lenient Default Unification for Robust Processing within Unification Based Grammar Formalisms Takashi NINOMIYA,†‡ Yusuke MIYAO,† and Jun’ichi TSUJII†‡ † Department of Computer Science, University of Tokyo ‡ CREST, Japan Science and Technology Corporation e-mail: {ninomi, yusuke, tsujii}@is.s.u-tokyo.ac.jp Abstract This paper describes new default unification, lenient default unification. It works efficiently, and gives more informative results because it maximizes the amount of information in the result, while other de- fault unification maximizes it in the default. We also describe robust processing within the framework of HPSG. We extract grammar rules from the results of robust parsing using lenient default unification. The results of a series of experiments show that parsing with the extracted rules works robustly, and the cov- erage of a manually-developed HPSG grammar for Penn Treebank was greatly increased with a little overgeneration. 1 Introduction Parsing has often been considered to be crucial for natural language processing, thus, efficient and wide coverage parsing has been extensively pur- sued in natural language literature. This study aims at robust processing within the Head-driven Phrase Structure Grammar (HPSG) to extend the cover- age of manually-developed HPSG grammars. The meaning of ‘robust processing’ is not limited to ro- bust processing for ill-formed sentences found in a spoken language, but includes robust processing for sentences which are well-formed but beyond the grammar writer’s expectation. Studies of robust parsing within unification-based grammars have been explored by many researchers (Douglas and Dale, 1992; Imaichi and Matsumoto, 1995). They classified the errors found in analyzing ill-formed sentences into several categories to make them tractable, e.g., constraint violation, missing or extra elements, etc. In this paper, we focus on re- covery from the constraint violation errors, which is a violation of feature values. All errors in agreement fall into this category. Since many of the grammat- ical components in HPSG are written as constraints represented by feature structures, many of the er- rors are expected to be recovered by the recovery of constraint violation errors. This paper proposes two new types of default unification and describes their application to robust processing. Default unification was originally stud- ied to develop a system of lexical semantics to deal with the default inheritance in a lexicon, but it is also desirable for the recovery of such constraint vi- olation errors due to the following merits: i) default unification is always well-defined, and ii) a feature structure is relaxed such that the amount of infor- mation is maximized. From the viewpoint of robust processing, an amount of lost information can be re- garded as a cost (i.e., penalty) of robust processing. In other words, default unification tries to minimize the cost. Given a strict feature structure F and a default feature structure G, default unification is de- fined as unification that satisfies the following (writ- ten as F < ⊔ G): 1) It is always defined. 2) All strict information is preserved. That is, F ⊑ (F < ⊔ G). 3) It reduces to standard unification in the case of F and …

Content truncated to 500 words