LIMSI: Learning Semantic Similarity by Selecting Random Word Subsets

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 543–546, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics LIMSI: Learning Semantic Similarity by Selecting Random Word Subsets Artem Sokolov LIMSI-CNRS B.P. 133, 91403 Orsay, France artem.sokolov@limsi.fr Abstract We propose a semantic similarity learning method based on Random Indexing (RI) and ranking with boosting. Unlike classical RI, we use only those context vector features that are informative for the semantics modeled. De- spite ignoring text preprocessing and dispens- ing with semantic resources, the approach was ranked as high as 22nd among 89 participants in the SemEval-2012 Task6: Semantic Textual Similarity. 1 Introduction One of the popular and flexible tools of semantics modeling are vector distributional representations of texts (also known as vector space models, seman- tic word spaces or distributed representations). The principle idea behind vector space models is to use word usage statistics in different contexts to gen- erate a high-dimensional vector representations for each word. Words are represented by context vec- tors whose closeness in the vector space is postu- lated to reflect semantic similarity (Sahlgren, 2005). The approach rests upon the distributional hypothe- sis: words with similar meanings or functions tend to appear in similar contexts. The prominent ex- amples of vector space models are Latent Seman- tic Analysis (or Indexing) (Landauer and Dutnais, 1997) and Random Indexing (Kanerva et al., 2000). Because of the heuristic nature of distributional methods, they are often designed with a specific semantic relation in mind (synonymy, paraphrases, contradiction, etc.). This complicates their adaption to other application domains and tasks, requiring manual trial-and-error feature redesigns and tailored preprocessing steps to remove morphology/syntax variations that are not supposed to contribute to the semantics facet in question (e.g., stemming, stop- words). Further, assessing closeness of semantic vectors is usually based on a fixed simple similarity function between distributed representations (often, the cosine function). The cosine function implicitly assigns equal weights to each component of the se- mantic vectors regardless of its importance for the particular semantic relation and task. Finally, dur- ing production of training and evaluation sets, the continuum of possible grades of semantic similar- ity is usually substituted with several integer values, although often only the relative grade order matters and not their absolute values. Trying to reproduce the same values or the same gaps between grades when designing a semantic representation scheme may introduce an unnecessary bias. In this paper we address all of the above draw- backs and present a semantic similarity learning method based on Random Indexing. It does not re- quire manual feature design, and is automatically adapted to the specific semantic relations by select- ing needed important features and/or learning neces- sary feature transformations before calculating sim- ilarity. In the proof-of-concept experiments on the SemEval-2012 data we deliberately ignored all rou- tine preprocessing steps, that are often considered obligatory in semantic text processing, we did not use any of the semantic resources (like WordNet) nor trained different models for different data do- mains/types. Despite such …

Content truncated to 500 words