A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 19–27, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches Eneko Agirre† Enrique Alfonseca‡ Keith Hall‡ Jana Kravalova‡§ Marius Pas¸ca‡ Aitor Soroa† † IXA NLP Group, University of the Basque Country ‡ Google Inc. § Institute of Formal and Applied Linguistics, Charles University in Prague {e.agirre,a.soroa}@ehu.es {ealfonseca,kbhall,mars}@google.com kravalova@ufal.mff.cuni.cz Abstract This paper presents and compares WordNet- based and distributional similarity approaches. The strengths and weaknesses of each ap- proach regarding similarity and relatedness tasks are discussed, and a combination is pre- sented. Each of our methods independently provide the best results in their class on the RG and WordSim353 datasets, and a super- vised combination of them yields the best pub- lished results on all datasets. Finally, we pio- neer cross-lingual similarity, showing that our methods are easily adapted for a cross-lingual task with minor losses. 1 Introduction Measuring semantic similarity and relatedness be- tween terms is an important problem in lexical se- mantics. It has applications in many natural lan- guage processing tasks, such as Textual Entailment, Word Sense Disambiguation or Information Extrac- tion, and other related areas like Information Re- trieval. The techniques used to solve this problem can be roughly classified into two main categories: those relying on pre-existing knowledge resources (thesauri, semantic networks, taxonomies or ency- clopedias) (Alvarez and Lim, 2007; Yang and Pow- ers, 2005; Hughes and Ramage, 2007) and those in- ducing distributional properties of words from cor- pora (Sahami and Heilman, 2006; Chen et al., 2006; Bollegala et al., 2007). In this paper, we explore both families. For the first one we apply graph based algorithms to Word- Net, and for the second we induce distributional similarities collected from a 1.6 Terabyte Web cor- pus. Previous work suggests that distributional sim- ilarities suffer from certain limitations, which make them less useful than knowledge resources for se- mantic similarity. For example, Lin (1998b) finds similar phrases like captive-westerner which made sense only in the context of the corpus used, and Budanitsky and Hirst (2006) highlight other prob- lems that stem from the imbalance and sparseness of the corpora. Comparatively, the experiments in this paper demonstrate that distributional similarities can perform as well as the knowledge-based approaches, and a combination of the two can exceed the per- formance of results previously reported on the same datasets. An application to cross-lingual (CL) sim- ilarity identification is also described, with applica- tions such as CL Information Retrieval or CL spon- sored search. A discussion on the differences be- tween learning similarity and relatedness scores is provided. The paper is structured as follows. We first present the WordNet-based method, followed by the distributional methods. Section 4 is devoted to the evaluation and results on the monolingual and cross- lingual tasks. Section 5 presents some analysis, in- cluding learning curves for distributional methods, the use of distributional similarity to improve Word- Net similarity, …

Content truncated to 500 words