A Corpus-Based Approach to Deriving Lexical Mappings

Proceedings of EACL '99 A Corpus-Based Approach to Deriving Lexical Mappings Mark Stevenson Department of Computer Science, University of Sheffield, Regent Court, 211 Portobello Street, Sheffield S1 4DP United Kingdom marks©dcs, shef. ac. uk Abstract This paper proposes a novel, corpus- based, method for producing mappings between lexical resources. Results from a preliminary experiment using part of speech tags suggests this is a promising area for future research. 1 Introduction Dictionaries are now commonly used resources in NLP systems. However, different lexical resources are not uniform; they contain different types of information and do not assign words the same number of senses. One way in which this prob- lem might be tackled is by producing mappings between the senses of different resources, the "dic- tionary mapping problem". However, this is a non-trivial problem, as examination of existing lexical resources demonstrates. Lexicographers have been divided between "lumpers', or those who prefer a few general senses, and "splitters" who create a larger number of more specific senses so there is no guarantee that a word will have the same number of senses in different resources. Previous attempts to create lexical mappings have concentrated on aligning the senses in pairs of lexical resources and based the mapping de- cision on information in the entries. For ex- ample, Knight and Luk (1994) merged WordNet and LDOCE using information in the hierarchies and textual definitions of each resource. Thus far we have mentioned only mappings between dictionary senses. However, it is possible to create mappings between any pair of linguistic annotation tag-sets; for example, part of speech tags. We dub the more general class lexical map- pings, mappings between two sets of lexical an- notations. One example which we shall consider further is that of mappings between part of speech tags sets. This paper shall propose a method for produ- cing lexical mappings based on corpus evidence. It is based on the existence of large-scale lexical an- notation tools such as part of speech taggers and sense taggers, several of which have now been de- veloped, for example (Brill, 1994)(Stevenson and Wilks, 1999). The availability of such taggers bring the possibility of automatically annotating large bodies of text. Our proposal is, briefly, to use a pair of taggers with each assigning annota- tions from the lexical tag-sets we are interested in mapping. These taggers can then be applied to, the same, large body of text and a mapping de- rived from the distributions of the pair of tag-sets in the corpus. 2 Case Study In order to test this approach we attempted to map together two part of speech tag-sets. We chose this form of linguistic annotation because it is commonly used in NLP systems and reliable taggers are readily available. The tags sets we shall examine are the set used in the Penn Tree Bank (PTB) (Marcus et al., 1993) and the C5 tag-set used by the CLAWS part-of-speech tagger (Garside, 1996). The PTB set consists of 48 annotations while the C5 uses a …

Content truncated to 500 words