Towards an optimal weighting of context words based on distance

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 107–115, Beijing, August 2010 Towards an optimal weighting of context words based on distance Bernard Brosseau-Villeneuve*#, Jian-Yun Nie*, Noriko Kando# * Université de Montréal, Email: {brosseab, nie}@iro.umontreal.ca # National Institute of Informatics, Email: {bbrosseau, kando}@nii.ac.jp Abstract Word Sense Disambiguation (WSD) of- ten relies on a context model or vector constructed from the words that co-occur with the target word within the same text windows. In most cases, a fixed-sized window is used, which is determined by trial and error. In addition, words within the same window are weighted uniformly regardless to their distance to the target word. Intuitively, it seems more reason- able to assign a stronger weight to con- text words closer to the target word. How- ever, it is difficult to manually define the optimal weighting function based on dis- tance. In this paper, we propose a unsu- pervised method for determining the op- timal weights for context words accord- ing to their distance. The general idea is that the optimal weights should maximize the similarity of two context models of the target word generated from two random samples. This principle is applied to both English and Japanese. The context mod- els using the resulting weights are used in WSD tasks on Semeval data. Our ex- perimental results showed that substantial improvements in WSD accuracy can be obtained using the automatically defined weighting schema. 1 Introduction The meaning of a word can be defined by the words that accompany it in the text. This is the principle often used in previous studies on Word Sense Disambiguation (WSD) (Ide and Véronis, 1998; Navigli, 2009). In general, the accompa- nying words form a context vector of the target word, or a probability distribution of the context words. For example, under the unigram bag-of- words assumption, this means building p(x|t) = count(x,t) � x′ count(x′,t), where count(x, t) is the count of co-occurrences of word x with the target word t under a certain criterion. In most studies, x and t should co-occur within a window of up to k words or sentences. The bounds are usually se- lected in an ad-hoc fashion to maximize system performance. Occurrences inside the window of- ten weight the same without regard to their po- sition. This is counterintuitive. Indeed, a word closer to the target word generally has a greater semantic constraint on the target word than a more distant word. It is however difficult to define the optimal weighting function manually. To get around this, some systems add positional features for very close words. In information retrieval, to model the strength of word relations, some studies have proposed non-uniform weighting methods of context words, which decrease the importance of more distant words in the context vector. How- ever, the weighting functions are defined manu- ally. It is unclear that these functions can best cap- ture the impact of the context words on the target word. In this paper, we propose an …

Content truncated to 500 words