Towards a Dictionary Support Environment for Realtime Parsing

TOWARDS A DICTIONARY SUPPORT ENVIRONMENT FOR REALTIME PARSING ABSTRACT Hiyan Alshawi, Bran Boguraev, Ted Briscoe Computer Laboratory, Cambridge University Corn Exchange Street Cambridge CB2 3QG, U.K. In this article we describe research on the development of large dictionaries for natural language processing. We detail the development of a dictionary support environment linking a restructrured version of the Longman Dictionary of Contemporary English to natural language processing systems. We describe the process of restructuring the information in the dictionary and our use of the Longman grammar code system to construct dictionary entries for the PATR-II parsing system and our use of the Longman word definitions for automated word sense classification. INTRODUCTION Recent developments in linguistics, and especially on grammatical theory - for example, Generalised Phrase Structure Grammar' (GPSG) (Gazdar et al., In Press), Lexical Functional Grammar (LFG) (Kaplan & Bresnan, 1982) - and on natural language parsing frameworks - for example, Functional Unification Grammar (FUG) (Kay, 1984a), PATR-II (Shieber, 1984) - make it feasible to consider the implementation of efficient systems for the syntactic analysis of substantial fragments of natural language. These developments also demonstrate that if natural language processing systems are to be able to handle the grammatical and logical idiosyncracies of individual lexical items elegantly and efficiently, then the lexicon must be a central component of the parsing system. Real-time parsing imposes stringent requirements on a dictionary support environment; at the very least it must allow frequent and rapid access to the information in the dictionary via the dictionary head words. The idea of using the machine-readable source of a published dictionary has occurred to a wide range of researchers - for spelling correction, lexical analysis, thesaurus construction, machine- translation, to name but a few applications - very few however have used such a dictionary to support a natural language parsing system. Most of the work on automated dictionaries has concentrated on extracting lexical or other information in, essentially, batch processing (eg. Amsler, 1981; Walker & Amsler, 1983), or on developing dictionary servers for office automation systems (Kay, 1984b). Few parsing systems have substantial lexicons and even those which employ very comprehensive grammars (eg. Robinson, 1982; Bobrow, 1978) consult relatively small lexicons, typically generated by hand. Two exceptions to this generalisation are the Linguistic String Project (Sager, 1981) and the Epistle Project (Heidorn et al., 1982); the former employs a dictionary of less than 10,000 words, most of which are specialist medical terms, the latter has well over 100,000 entries, gathered from machine-readable sources, however, their grammar formalism and the limited grammatical information supplied by the dictionary make this achievement, though impressive, theoretically less interesting. We chose to employ the Longman Dictionary of Contemporary English (Procter 1978, henceforth LDOCE) as the machine-readable source for our dictionary environment because this dictionary has several properties which make it uniquely appropriate for use as the core knowledge base of a natural language processing system. Most prominent among these are the rich grammatical subcategorisations of the 60,000 entries, the large amount of information …

Content truncated to 500 words