Estimating Class Priors in Domain Adaptation for Word Sense Disambiguation

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 89–96, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Estimating Class Priors in Domain Adaptation for Word Sense Disambiguation Yee Seng Chan and Hwee Tou Ng Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore 117543 � chanys,nght � @comp.nus.edu.sg Abstract Instances of a word drawn from different domains may have different sense priors (the proportions of the different senses of a word). This in turn affects the accuracy of word sense disambiguation (WSD) sys- tems trained and applied on different do- mains. This paper presents a method to estimate the sense priors of words drawn from a new domain, and highlights the im- portance of using well calibrated probabil- ities when performing these estimations. By using well calibrated probabilities, we are able to estimate the sense priors effec- tively to achieve significant improvements in WSD accuracy. 1 Introduction Many words have multiple meanings, and the pro- cess of identifying the correct meaning, or sense of a word in context, is known as word sense disambiguation (WSD). Among the various ap- proaches to WSD, corpus-based supervised ma- chine learning methods have been the most suc- cessful to date. With this approach, one would need to obtain a corpus in which each ambiguous word has been manually annotated with the correct sense, to serve as training data. However, supervised WSD systems faced an important issue of domain dependence when using such a corpus-based approach. To investigate this, Escudero et al. (2000) conducted experiments using the DSO corpus, which contains sentences drawn from two different corpora, namely Brown Corpus (BC) and Wall Street Journal (WSJ). They found that training a WSD system on one part (BC or WSJ) of the DSO corpus and applying it to the other part can result in an accuracy drop of 12% to 19%. One reason for this is the difference in sense priors (i.e., the proportions of the different senses of a word) between BC and WSJ. For in- stance, the noun interest has these 6 senses in the DSO corpus: sense 1, 2, 3, 4, 5, and 8. In the BC part of the DSO corpus, these senses occur with the proportions: 34%, 9%, 16%, 14%, 12%, and 15%. However, in the WSJ part of the DSO cor- pus, the proportions are different: 13%, 4%, 3%, 56%, 22%, and 2%. When the authors assumed they knew the sense priors of each word in BC and WSJ, and adjusted these two datasets such that the proportions of the different senses of each word were the same between BC and WSJ, accuracy im- proved by 9%. In another work, Agirre and Mar- tinez (2004) trained a WSD system on data which was automatically gathered from the Internet. The authors reported a 14% improvement in accuracy if they have an accurate estimate of the sense pri- ors in the evaluation data and sampled their train- ing data …

Content truncated to 500 words