Automatic Acquisition of a Large Sub Categorization Dictionary From Corpora

AUTOMATIC ACQUISITION OF A LARGE SUBCATEGORIZATION DICTIONARY FROM CORPORA Christopher D. Manning Xerox PARC and Stanford University Stanford University Dept. of Linguistics, Bldg. 100 Stanford, CA 94305-2150, USA Internet: manning@csli.stanford.edu Abstract This paper presents a new method for producing a dictionary of subcategorization frames from un- labelled text corpora. It is shown that statistical filtering of the results of a finite state parser run- ning on the output of a stochastic tagger produces high quality results, despite the error rates of the tagger and the parser. Further, it is argued that this method can be used to learn all subcategori- zation frames, whereas previous methods are not extensible to a general solution to the problem. INTRODUCTION Rule-based parsers use subcategorization informa- tion to constrain the number of analyses that are generated. For example, from subcategorization alone, we can deduce that the PP in (1) must be an argument of the verb, not a noun phrase mod- ifier: (1) John put [Nethe cactus] [epon the table]. Knowledge of subcategorization also aids text ger- eration programs and people learning a foreign language. A subcategorization frame is a statement of what types of syntactic arguments a verb (or ad- jective) takes, such as objects, infinitives, that- clauses, participial clauses, and subcategorized prepositional phrases. In general, verbs and ad- jectives each appear in only a small subset of all possible argument subcategorization frames. A major bottleneck in the production of high- coverage parsers is assembling lexical information, °Thanks to Julian Kupiec for providing the tag- ger on which this work depends and for helpful dis- cussions and comments along the way. I am also indebted for comments on an earlier draft to Marti Hearst (whose comments were the most useful!), Hin- rich Schfitze, Penni Sibun, Mary Dalrymple, and oth- ers at Xerox PARC, where this research was completed during a summer internship; Stanley Peters, and the two anonymous ACL reviewers. such as subcategorization information. In early and much continuing work in computational lin- guistics, this information has been coded labori- ously by hand. More recently, on-line versions of dictionaries that provide subcategorization in- formation have become available to researchers (Hornby 1989, Procter 1978, Sinclair 1987). But this is the same method of obtaining subcatego- rizations - painstaking work by hand. We have simply passed the need for tools that acquire lex- ical information from the computational linguist to the lexicographer. Thus there is a need for a program that can ac- quire a subcategorization dictionary from on-line corpora of unrestricted text: 1. Dictionaries with subcategorization information are unavailable for most languages (only a few recent dictionaries, generally targeted at non- native speakers, list subcategorization frames). 2. No dictionary lists verbs from specialized sub- fields (as in I telneted to Princeton), but these could be obtained automatically from texts such as computer manuals. 3. Hand-coded lists are expensive to make, and in- variably incomplete. 4. A subcategorization dictionary obtained auto- matically from corpora can be updated quickly and easily as different usages develop. Diction- aries …

Content truncated to 500 words