Linguistic Knowledge Acquisition from Parsing Failures

Linguistic Knowledge Acquisition from Parsing Failures Masaki KIYONO* and Jun-ichi TSUJII (kiyono@ccl.umist.ac.uk and tsujii@ccl.umist.a~.uk) Centre for Computational Linguistics University of Manchester Institute of Science and Technology PO Box 88, Manchester M60 1QD United Kingdom Abstract A semi-automatic procedure of linguistic knowledge acquisition is proposed, which combines corpus-based techniques with the conventional rule-based approach. The rule-based component generates all the pos- sible hypotheses of defects which the ex- isting linguistic knowledge might contain, when it fails to parse a sentence. The rule-based component does not try to iden- tify the defects, but generates a set of hy- potheses and the corpus-based component chooses the plausible ones among them. The procedure will be used for adapting or re-using existing linguistic resources for new application domains. 1 Introduction While quite a number of useful grammar formalisms for natural language processing now exist, it still re- mains a time-consuming and hard task to develop grammars and dictionaries with comprehensive cov- erage. It is also the case that, though quite a few computational grammars and dictionaries with com- prehensive coverage have been used in various ap- plication systems, to re-use them for other applica- tion domains is not always so easy, even if we use the same formalisms and programs such as parsers, etc. We usually have to revise, add, and delete grammar rules and lexical entries in order to adapt them to the peculiarities of languages (sublanguages) of new application domains [Sekine et al., 1992; Tsujii et al., 1992; Ananiadou, 1990]. *also a staff member of Matsushita Electric Industrial Co.,Ltd., Tokyo, JAPAN. Such adaptations of existing linguistic knowledge to a new domain are currently performed through rather undisciplined, trial and error processes in- volving much human effort. In this paper we show that techniques similar to those in robust parsing of ill-formed input, together with corpus-based tech- niques, can be used to discover disparities between existing linguistic knowledge and actual language us- age in a new domain, and to hypothesize new gram- mar rules or lexical descriptions. Although our framework appears similar to gram- mar learning from corpora, our current goal is far more modest, i.e. to help linguists revise existing grammars by showing possible defects and hypothe- sizing them through corpus analysis. 2 Robust Parsing and Linguistic Knowledge Acquisition 2.1 Search Space of Possible Hypotheses When a parser fails to analyse an input sentence, a robust parser hypothesizes possible errors in the input in order to complete the analysis and correct errors [Douglas and Dale, 1992]: for example, dele- tion of necessary words (Ex. I have book), insertion of unnecessary words (Ex. I have a the book), dis- order of words (Ex. I a book have), spelling errors (Ex. I have a bok), etc. As there is usually a set of possible hypotheses to complete the analysis, this error detection process becomes non-deterministic. Furthermore, allowing operations such as deletion and insertion of arbi- trary sequences of words or unrestricted permuta- tion of word sequences, radically expands its search space. The process generates many …

Content truncated to 500 words