Lexical ambiguity and Information Retrieval revisited

Lexical ambiguity and Information Retrieval revisited Julio Gonzalo Anselmo Pefias Felisa Verdejo UNED Ciudad Universitaria, s.n. 28040 Madrid - Spain {j ulio, anselmo, f elisa}@ieec, uned. es Abstract A number of previous experiments on the role of lexical ambiguity, in Information Retrieval are re- produced on the'IR-Semcor test collection (derived from Semcor), where both queries and documents are hand-tagged ;with phrases, Part-Of-Speech and WordNet 1.5 senses. Our results indicate that a) Word Sense Disambigua- tion can be more beneficial to Information Retrieval than the experiments of Sanderson (1994) with arti- ficially ambiguous pseudo-words suggested, b) Part- Of-Speech tagging does not seem to help Improving retrieval, even if it is manually annotated, c) Using phrases as indexing terms is not a good strategy if no partial credit is given to the phrase components. 1 Introduction A major difficulty to experiment with lexical am- biguity issues in Information Retrieval is always to differentiate the effects of the indexing and retrieval strategy being tested from the effects of tagging er- rors. Some examples are: 1. In (RichardSon and Smeaton, 1995), a sophisti- cated retrieval system based on conceptual sim- ilarity resultled in a decrease of IR performance. It was not possible, however, to distinguish the effects of the strategy and the effects of auto- matic Wordl Sense Disambiguation (WSD) er- rors. In (Smeaton and Quigley, 1996), a simi- lar strategy and a combination of manual dis- ambiguation and very short documents -image captions- pioduced, however, an improvement of IR perforinance. 2. In (Krovetz, 1997), discriminating word senses with differefit Part-Of-Speech (as annotated by the Church :POS tagger) also harmed retrieval efficiency. Krovetz noted than more than half of the words in a dictionary that differ in POS are related i n meaning, but he could not decide whether the decrease of performance was due to the loss of such semantic relatedness or to automatic POS tagging errors. 3. In (Sanderson, 1994), the problem of dis- cerning the effects of differentiating word senses from the effects of inaccurate dis- ambiguation was overcome using artificially created pseudo-words (substituting, for in- stance, all occurrences of banana or kalashnikov for banana/kalashnikov) that could be disam- biguated with 100% accuracy (substituting ba- nana/kalashnikov back to the original term in each occurrence, either banana or kalashnikov). He found that IR processes were quite resistant to increasing degrees of lexical ambiguity, and that disambiguation harmed IR efficiency if per- formed with less that 90% accuracy. The ques- tion is whether real ambiguous words would be- have as pseudo-words. 4. In (Schiitze and Pedersen, 1995) it was shown that sense discriminations extracted from the test collections may enhance text retrieval. However, the static sense inventories in dictio- naries or thesauri -such as WordNet- have not been used satisfactorily in IR. For instance, in (Voorhees, 1994), manual expansion of TREC queries with semantically related words from WordNet only produced slight improvements with the shortest queries. In order to deal with these problems, we designed an IR test collection which is hand annotated …

Content truncated to 500 words