RALI: Automatic Weighting of Text Window Distances

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 375–378, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics RALI: Automatic weighting of text window distances Bernard Brosseau-Villeneuve*#, Noriko Kando#, Jian-Yun Nie* * Université de Montréal, Email: {brosseab, nie}@iro.umontreal.ca # National Institute of Informatics, Email: {bbrosseau, kando}@nii.ac.jp Abstract Systems using text windows to model word contexts have mostly been using fixed-sized windows and uniform weights. The window size is often selected by trial and error to maximize task results. We propose a non-supervised method for se- lecting weights for each window distance, effectively removing the need to limit win- dow sizes, by maximizing the mutual gen- eration of two sets of samples of the same word. Experiments on Semeval Word Sense Disambiguation tasks showed con- siderable improvements. 1 Introduction The meaning of a word can be defined by the words that accompany it in the text. This is the principle often used in previous studies on Word Sense Disambiguation (WSD) (Ide and Véronis, 1998; Navigli, 2009). In general, the accompa- nying words form a context vector of the target word, or a probability distribution of the context words. For example, under the unigram bag-of- word assumption, this means building p(x|t) = count(x,t) ∑ x′ count(x′,t), where count(x, t) is the count of co-occurrences of word x with the target word t under a certain criterion. In most studies, x and t should co-occur within a window of up to k words or sentences. The bounds are usually selected as to maximize system performance. Occurrences in- side the window usually weight the same with- out regard to their position. This is counterintu- itive. Indeed, a word closer to the target word usu- ally has a greater semantic constraint on the tar- get word than a more distant word. Some studies have also proposed decaying factors to decrease the importance of more distant words in the con- text vector. However, the decaying functions are defined manually. It is unclear that the functions defined can capture the true impact of the con- text words on the target word. In this paper, we propose an unsupervised method to automatically learn the optimal weight of a word according to its distance to the target word. The general idea used to determine such weight is that, if we randomly determine two sets of texts containing the target word, the resulting probability distributions for its context words in the two sets should be similar. Therefore, the weights of context words at differ- ent distance are determined so as to maximize the mutual generation probabilities of two sets of sam- ples. Experimentation on Semeval-2007 English and Semeval-2010 Japanese lexical sample task data shows that improvements can automatically be attained on simple Naive Bayes (NB) systems in comparison to the best manually selected fixed window system. The remainder of this paper is organized as fol- lows: example uses of text windows and related work are presented in Section 2. Our method is presented in Section …

Content truncated to 500 words