Combining Heterogeneous Models for Measuring Relational Similarity

Proceedings of NAACL-HLT 2013, pages 1000–1009, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Combining Heterogeneous Models for Measuring Relational Similarity Alisa Zhila∗ Instituto Politecnico Nacional Mexico City, Mexico alisa.zhila@gmail.com Wen-tau Yih Christopher Meek Microsoft Research Redmond, WA 98052, USA {scottyih,meek}@microsoft.com Geoffrey Zweig Microsoft Research Redmond, WA 98052, USA gzweig@microsoft.com Tomas Mikolov∗ BRNO University of Technology BRNO, Czech Republic tmikolov@gmail.com Abstract In this work, we study the problem of mea- suring relational similarity between two word pairs (e.g., silverware:fork and clothing:shirt). Due to the large number of possible relations, we argue that it is important to combine mul- tiple models based on heterogeneous informa- tion sources. Our overall system consists of two novel general-purpose relational similar- ity models and three specific word relation models. When evaluated in the setting of a recently proposed SemEval-2012 task, our ap- proach outperforms the previous best system substantially, achieving a 54.1% relative in- crease in Spearman’s rank correlation. 1 Introduction The problem of measuring relational similarity is to determine the degree of correspondence between two word pairs. For instance, the analogous word pairs silverware:fork and clothing:shirt both exem- plify well a Class-Inclusion:Singular Collective re- lation and thus have high relational similarity. Un- like the problem of attributional similarity, which measures whether two words share similar attributes and is addressed in extensive research work (Bu- danitsky and Hirst, 2006; Reisinger and Mooney, 2010; Radinsky et al., 2011; Agirre et al., 2009; Yih and Qazvinian, 2012), measuring relational similar- ity is a relatively new research direction pioneered by Turney (2006), but with many potential appli- cations. For instance, problems of identifying spe- cific relations between words, such as synonyms, ∗Work conducted while interning at Microsoft Research. antonyms or associations, can be reduced to mea- suring relational similarity compared to prototypical word pairs with the desired relation (Turney, 2008). In scenarios like information extraction or question answering, where identifying the existence of cer- tain relations is often the core problem, measuring relational similarity provides a more flexible solu- tion rather than creating relational classifiers for pre- defined or task-specific categories of relations (Tur- ney, 2006; Jurgens et al., 2012). In order to promote this research direction, Ju- rgens et al. (2012) proposed a new shared task of measuring relational similarity in SemEval-2012 re- cently. In this task, each submitted system is re- quired to judge the degree of a target word pair having a particular relation, measured by its re- lational similarity compared to a few prototypical example word pairs. The system performance is evaluated by its correlation with the human judg- ments using two evaluation metrics, Spearman’s rank correlation and MaxDiff accuracy (more de- tails of the task and evaluation metrics will be given in Sec. 3). Although participating systems incorpo- rated substantial amounts of information from lex- ical resources (e.g., WordNet) and contextual pat- terns from large corpora, only one system (Rink and Harabagiu, 2012) is able to outperform a simple baseline that uses PMI (pointwise mutual informa- tion) scoring, which demonstrates the …

Content truncated to 500 words