Feature Vector Quality and Distributional Similarity

Feature Vector Quality and Distributional Similarity Maayan Geffet School of Computer Science and Engineering, Hebrew University Givat Ram Campus, Jerusalem, Israel, 91904 mary@cs.huji.ac.il Ido Dagan Department of Computer Science, Bar-Ilan University Ramat-Gan, Israel, 52900 dagan@cs.biu.ac.il Abstract We suggest a new goal and evaluation criterion for word similarity measures. The new criterion - meaning-entailing substitutability - fits the needs of semantic-oriented NLP applications and can be evaluated directly (independent of an application) at a good level of human agreement. Motivated by this semantic criterion we analyze the empirical quality of distributional word feature vectors and its impact on word similarity results, proposing an objective measure for evaluating feature vector quality. Finally, a novel feature weighting and se- lection function is presented, which yields superior feature vectors and better word similarity perform- ance. 1 Introduction Distributional Similarity has been an active re- search area for more than a decade (Hindle, 1990), (Ruge, 1992), (Grefenstette, 1994), (Lee, 1997), (Lin, 1998), (Dagan et al., 1999), (Weeds and Weir, 2003). Inspired by Harris distributional hy- pothesis (Harris, 1968), similarity measures com- pare a pair of weighted feature vectors that characterize two words. Features typically corre- spond to other words that co-occur with the charac- terized word in the same context. It is then assumed that different words that occur within similar contexts are semantically similar. As it turns out, distributional similarity captures a somewhat loose notion of semantic similarity (see Table 1). By construction, if two words are distributionally similar then the occurrence of one word in some contexts indicates that the other word is also likely to occur in such contexts. But it does not ensure that the meaning of the first word is preserved when replacing it with the other one in the given context. For example, words of similar semantic types, such as company – government, tend to come up as distributionally similar, even though they are not substitutable in a meaning pre- serving sense. On the other hand, many semantic-oriented appli- cations, such as Question Answering, Paraphrasing and Information Extraction, do need to recognize which words may substitute each other in a mean- ing preserving manner. For example, a question about company may be answered by a sentence about firm, but not about government. Such appli- cations usually utilize reliable taxonomies or on- tologies like WordNet, but cannot rely on the “loose” type of output of distributional similarity measures. In recent work Dagan and Glickman (2004) ob- serve that applications usually do not require a strict meaning preserving criterion between text expressions, but rather need to recognize that the meaning of one expression entails the other. En- tailment modeling is thus proposed in their work as a generic (application-independent) framework for practical semantic inference. We suggest adopting such (directional) entailment criterion at the lexical level for judging whether one word can be substi- tuted by another one. For example, certain ques- tions about companies might be answered by sentences about automakers, since the meaning of automaker entails the meaning of company …

Content truncated to 500 words