Extracting Semantic Hierarchies From a Large On-Line Dictionary

EXTRACTING SEMANTIC HIERARCHIES FROM A LARGE ON-LINE DICTIONARY Martin S. Chodorow Department of Psychology, Hunter College of CUNY and I.B.M. Thomas J. Watson Research Center Yorktown Heights, New York 10598 Roy J. Byrd George E. Heidorn I.B.M. Thomas J. Watson Research Center Yorktown Heights, New York 10598 ABSTRACT Dictionaries are rich sources of detailed semantic infor- mation, but in order to use the information for natural language processing, it must be organized systematically. This paper describes automatic and semi-automatic procedures for extracting and organizing semantic fea= ture information implicit in dictionary definitions. Two head-finding heuristics are described for locating the genus terms in noun and verb definitions. The assump- tion is that the genus term represents inherent features of the word it defines. The two heuristics have been used to process definitions of 40,000 nouns and 8,000 verbs, producing indexes in which each genus term is associated with the words it defined. The Sprout pro- gram interactively grows a taxonomic "tree" from any specified root feature by consulting the genus index. Its output is a tree in which all of the nodes have the root feature for at least one of their senses. The Filter pro- gram uses an inverted form of the genus index. Filtering begins with an initial filter file consisting of words that have a given feature (e.g. [+human]) in all of their senses. The program then locates, in the index, words whose genus terms all appear in the filter file. The out- put is a list of new words that have the given feature in all of their senses. 1. Introduction. The goal of this research is to extract semantic informa- tion from standard dictionary definitions, for use in constructing lexicons for natural language processing systems. Although dictionaries contain finely detailed semantic knowledge, the systematic organization of that knowledge has not heretofore been exploited in such a way as to make the information available for computer applications. Amsler(1980) demonstrates that additional structure can be imposed upon a dictionary by making certain as- sumptions about the ways in which definitions are con- structed. Foremost among these assumptions is that definitions consist of a "genus" term, which identifies the superordinate concept of the defined word, and "differentia" which distinguish this instance of the superordinate category from other instances. By manu- ally extracting and disambiguating genus terms for a pocket dictionary, Amsler demonstrated the feasibility of generating semantic hierarchies. It was our goal to automate the genus extraction and disambiguation processes so that semantic hierarchies could be generated from full-sized dictionaries. The fully automatic genus extraction process is described in Section 2. Sections 3 and 4 describe two different disambiguation and hierarchy-extraction techniques that rely on the genus information. Both of these techniques 299 are semi-automatic, since they crucially require decisions • to be made by a human user during processing. Never- theless, significant savings occur when the system or- ganizes the presentation of material to the user. Further economy results from the automatic access to word de- finitions …

Content truncated to 500 words