Using a Semantic Concordance for Sense Identification

USING A SEMANTIC CONCORDANCE FOR SENSE IDENTIFICATION George A. Miller, Martin Chodorow*, Shari Landes, Claudia Leacock, and Robert G. Thomas Cognitive Science Laboratory Princeton University Princeton, NJ 08542 ABSTRACT This paper proposes benchmarks for systems of automatic sense identification. A textual corpus in which open-class words had been tagged both syntactically and semantically was used to explore three statistical strategies for sense identification: a guess- ing heuristic, a most-frequent heuristic, and a co-occurrence heuris- tic. When no information about sense-frequencies was available, the guessing heuristic using the numbers of alternative senses in WordNet was correct 45% of the time. When statistics for sense- frequancies were derived from a semantic concordance, the assumption that each word is used in its most frequently occurring sense was correct 69% of the time; when that figure was calculated for polysemous words alone, it dropped to 58%. And when a co- occur~nce heuristic took advantage of prior occurrences of words together in the same sentences, little improvement was observed. The semantic concordance is still too small to estimate the potential limits of a co-occurrence heuristic. 1. INTRODUCTION It is generally recognized that systems for automatic seine identification should be evaluated against a null hypothesis. Gale, Church, and Yarowsky [1] suggest that the appropriate basis for comparison would be a system that assumes that each word is being used in its most frequently occurring sere. They review the litera- ture on how well word-disambiguation programs perform; as a lower bound, they estimate that the most frequent sense of polysemous words would be correct 75% of the time, and they pro- pose that any sense-identification system that does not give the correct sense of polysemous words more than 75% of the time would not be worth serious consideration. The value of setting such a lower bound is obvious. However, Gale" Church, and Yarowsky [I] do not make clear how they deter- mined what the most frequently occurring senses are. In the absence of such information, a case can be made that the lower bound should be given by the proportion of monosemous words in the textual corpus. Although most words in a dictionary have only a single sense" it is the polysemons words that occur most frequently in speech and writing. This is true even when we ignore the small set of highly pelysemous closed-class words (pronouns, prepositions, auxiliary verbs, etc.) that play such an important structural role. For exam- pie, 82.3% of the opon-class words in WordNet [2] are monosemous, but only 27.2% of the open-class words in a sample of 103 passages from the Brown Corpus [3] were monosemous. * Hunter College and Graduate School of the City Univendty of New Ytz~k That is to say, 27% of the time no decision would be needed, but for the remaining 73% of the open-class words, the response would have to be "don't know." This is probably the lowest lower bound anyone would propose, although if the highly pelysemous, very fre- quently used closed-class words were included, …

Content truncated to 500 words