Determining the specificity of nouns from text

I Determining the specificity of nouns from text Sharon A. Caraballo and Eugene Charniak Dept. of Computer Science Brown University Providence, RI 02912 { SC, ec}@cs, brown, edu Abstract In this work, we use a large text corpus to order nouns by their level of specificity. This semantic information can for most nouns be determined with over 80% accuracy using simple statistics from a text corpus with- out using any additional sources of seman- tic knowledge. This kind of semantic in- formation can be used to help in automat- ically constructing or augmenting a lexical database such as WordNet. 1 Introduction Large lexical databases such as Word- Net (see Fellbaum (1998)) are in com- mon research use. However, there are cir- cumstances, particularly involving domain- specific text, where WordNet does not have sufficient coverage. Various automatic meth- ods have been proposed to automatically build lexical resources or augment existing resources. (See, e.g., Riloff and Shepherd (1997), Roark and Charniak (1998), Cara- hallo (1999), and Berland and Charniak (1999).) In this paper, we describe a method which can be used to assist in this problem. We present here a way to determine the relative specificity of nouns; that is, which nouns are more specific (or more general) than others, using only a large text cor- pus and no additional sources of semantic knowledge. By gathering simple statistics, we are able to decide which of two nouns is more specific to over 80% accuracy for nouns at "basic level" or below (see, e.g., Lakoff (1987)), and about 59% accuracy for nouns above basic level. It should be noted that specificity by it- self is not enough information from which to construct a noun hierarchy. This project is meant to provide a tool to support other methods. See Caraballo (1999) for a detailed description of a method to construct such a hierarchy. 2 Previous work To the best of our knowledge, this is the first attempt to automatically rank nouns based on specificity. Hearst (1992) found individual pairs of hypernyms and hyponyms from text using pattern-matching techniques. The sparse- ness of these patterns prevents this from be- ing an effective approach to the problem we address here. In Caraballo (1999), we construct a hierar- chy of nouns, including hypernym relations. However, there are several areas where that work could benefit from the research pre- sented here. The hypernyms used to label the internal nodes of that hierarchy are cho- sen in a simple fashion; pattern-matching as in Hearst (1992) is used to identify candi- date hypernyms of the words dominated by a particular node, and a simple voting scheme selects the hypernyms to be used. The hy- pernyms tend to lose quality as one looks closer to the root of the tree, generally by being too specific. This work could help to choose more general hypernyms from among the candidate words. In addition, it could be used to correct places where a more spe- cific word is listed as a hypernym of a …

Content truncated to 500 words