Measuring Topic Homogeneity and its Application to Dictionary-Based Word Sense Disambiguation

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 273–280 Manchester, August 2008 Measuring Topic Homogeneity and its Application to Dictionary-based Word Sense Disambiguation Ann Gledson and John Keane School of Computer Science, University of Manchester Oxford Road, Manchester, UK M13 9PL {ann.gledson,john.keane}@manchester.ac.uk Abstract The use of topical features is abundant in Natural Language Processing (NLP), a major example being in dictionary-based Word Sense Disambiguation (WSD). Yet previous research does not attempt to measure the level of topic cohesion in documents, despite assertions of its ef- fects. This paper introduces a quantitative measure of Topic Homogeneity using a range of NLP resources and not requiring prior knowledge of correct senses. Eval- uation is performed firstly by using the WordNet::Domains package to create word-sets with varying levels of homo- geneity and comparing our results with those expected. Additionally, to evaluate each measure’s potential value, the ho- mogeneity results are correlated against those of 3 co-occurrence/dictionary- based WSD techniques, tested on 1040 Semcor and SENSEVAL sub-documents. Many low-moderate correlations are found to exist with several in the mod- erate range (above .40). These correla- tions surpass polysemy and sense- entropy, the 2 most cited factors affecting WSD. Finally, a combined homogeneity measure achieves correlations of up to .52. 1 Introduction Topical features in NLP consist of unordered bags of words, often the context of a target word or phrase. In WSD for example, the word bank in the sentence: ‘If you’re OK being tied to one © 2008. Licensed under the Creative Commons Attri- bution-Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-nc- sa/3.0/). Some rights reserved. bank, you can get all your financial products there.’ might be assigned its monetary sense, based on the occurrence of the term financial. Often referred to as ‘topical features’, these are an important part of many NLP methods, such as WSD (Yarowsky 1995) and Topic Area Detec- tion (TAD) (Hearst 1997). Furthermore, in the SENSEVAL WSD competitions they are in- cluded in the highest performing systems. We assert that the effectiveness of topical fea- tures in NLP depends upon the level of topic homogeneity in the text. To illustrate two ex- tremes: the disambiguation of the word bank might be more difficult if occurring in i) a work of fiction describing a series of activities which includes the phrases: ‘stroll along the river’ and ‘pick up her cheque book’; than in ii) a news re- port on a bank in financial difficulty (a topically homogeneous text). This paper contributes a set of unsupervised Topic Homogeneity measures requiring no knowledge of correct senses. A variety of NLP resources are utilized and a set of evaluation me- thods devised, providing useful results. The pa- per is structured as follows: Section 2 outlines related work; Section 3 describes the experi- ments focusing on the resources used and their associated homogeneity measures; in Section 4 three evaluation methods are described, includ- ing a WSD task-based evaluation; Conclusions and future work are presented in Section 5. 2 Related Work TAD …

Content truncated to 500 words