Sense Tagging in Action Combining Different Tests with Additive Weighangs

Sense Tagging in Action Combining Different Tests with Additive Weightings Andrew Harley & Dominic Glennon Cambridge Language Services Ltd 64 Baldock Street Ware Herts SG12 9DT England andrew @ oaldeaf.demon.co.nk Abstract This paper describes a working sense tagger, which attempts to automatically link each word in a text corpus to its corresponding sense in a machine- readable dictionary. It uses information automatically extracted from the MRD to find matches between the dictionary and the Corpus sentences, and combines different types of information by simple additive scores with manually set weightings. 1. Introduction This paper describes a working sense tagger, which attempts to automatically link each word in a text corpus to its corresponding sub-sense in the Cambridge International Dictionary of English (CIDE). Much research elsewhere has gone into the generation of probabilities from corpora and the extraction of textual information from printed dictionaries. Our research has had the distinct advantage of being done alongside a large lexicographic team, who have been developing further the database used for the creation of CIDE. It has thus been possible to have very useful computational data expertly coded by hand. We have been able to concentrate on defining the specification of this lexical resource, encoding it and then making use of it, rather than on trying to extract or refine the desired information automatically from existing corpora or printed dictionaries. 2. Methodology The tagger, at present, works on one sentence at a time. Each word in the sentence has a certain number of possible senses. The tagger assigns a score (initially 0) to each possible sense of each word. A number of different tagging process could then adjust any of these scores, increasing them for a positive match (e.g. a collocation that indicates a particular sense), decreasing them for a negative match (e.g. capitalisation indicating a particular sense to be unlikely). At the end of all these processes, each sense of each word will have a particular score. For each word, the sense with the highest score is assumed to be the sense meant in the context. Simple additive weightings are also commonly used in the evaluation of chess positions by computers, where for example, a pawn less could score -100 and an open file for a rook +15. It is thus possible for a number of positional factors to outweigh more concrete material factors. It would be possible to use multiplicative probabilities rather than additive weightings. Chess programmers tend to prefer additive weightings because they ate far simpler to program and also more efficient. There are more rigorous rules for combining probabilities, but it is not clear how much benefit this gives if the original probabilities are only rough estimates anyway. Probabilities can be derived from training corpora, but it is acknowledged that these can vary enormously from corpus to corpus, e.g. on grounds of register (Biber 1993). Such methods are far more appropriate for work in restricted contexts, where representative training corpora can be more easily derived. 3. ProcJ~ure Besides some simple …

Content truncated to 500 words