Measuring Word Relatedness Using Heterogeneous Vector Space Models

2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 616–620, Montr´eal, Canada, June 3-8, 2012. c⃝2012 Association for Computational Linguistics Measuring Word Relatedness Using Heterogeneous Vector Space Models Wen-tau Yih Microsoft Research One Microsoft Way Redmond, WA scottyih@microsoft.com Vahed Qazvinian∗ Department of EECS University of Michigan Ann Arbor, MI vahed@umich.edu Abstract Noticing that different information sources of- ten provide complementary coverage of word sense and meaning, we propose a simple and yet effective strategy for measuring lexical se- mantics. Our model consists of a committee of vector space models built on a text cor- pus, Web search results and thesauruses, and measures the semantic word relatedness us- ing the averaged cosine similarity scores. De- spite its simplicity, our system correlates with human judgements better or similarly com- pared to existing methods on several bench- mark datasets, including WordSim353. 1 Introduction Measuring the semantic relatedness of words is a fundamental problem in natural language process- ing and has many useful applications, including textual entailment, word sense disambiguation, in- formation retrieval and automatic thesaurus discov- ery. Existing approaches can be roughly catego- rized into two kinds: knowledge-based and corpus- based, where the former includes graph-based algo- rithms and similarity measures operating on a lexical database such as WordNet (Budanitsky and Hirst, 2006; Agirre et al., 2009) and the latter consists of various kinds of vector space models (VSMs) constructed with the help of a large collection of text (Reisinger and Mooney, 2010; Radinsky et al., 2011). In this paper, we present a conceptually simple model for solving this problem. Observing that various kinds of information sources, such as ∗Work conducted while interning at Microsoft Research. general text corpora, Web search results and the- sauruses, have different word and sense coverage, we first build individual vector space models from each of them separately. Given two words, each VSM measures the semantic relatedness by the co- sine similarity of the corresponding vectors in its space. The final prediction is simply the averaged cosine scores derived from these VSMs. Despite its simplicity, our system surprisingly yields very strong empirical performance. When comparing the predictions with the human annotations on four dif- ferent datasets, our system achieves higher correla- tion than existing methods on two datasets and pro- vides very competitive results on the others. The rest of this paper is organized as follows. Sec- tion 2 briefly reviews the related work. Section 3 de- tails how we construct each individual vector space model, followed by the experimental evaluation in Section 4. Finally, Section 5 concludes the paper. 2 Background Prior work on measuring lexical semantics can be categorized as knowledge-based or corpus-based. Knowledge-based methods leverage word relations encoded in lexical databases such as WordNet and provide graph-based similarity measures. Detailed comparisons of these methods can be found in (Bu- danitsky and Hirst, 2006). Corpus-based methods assume related words tend to co-occur or to ap- pear in similar context. For example, Gabrilovich and Markovitch (2007) measure word relatedness …

Content truncated to 500 words