UTD: Determining Relational Similarity Using Lexical Patterns

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 413–418, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics UTD: Determining Relational Similarity Using Lexical Patterns Bryan Rink and Sanda Harabagiu University of Texas at Dallas P.O. Box 830688; MS EC31 Richardson, TX, 75083-0688, USA {bryan,sanda}@hlt.utdallas.edu Abstract In this paper we present our approach for assigning degrees of relational similarity to pairs of words in the SemEval-2012 Task 2. To measure relational similarity we employed lexical patterns that can match against word pairs within a large corpus of 12 million docu- ments. Patterns are weighted by obtaining sta- tistically estimated lower bounds on their pre- cision for extracting word pairs from a given relation. Finally, word pairs are ranked based on a model predicting the probability that they belong to the relation of interest. This ap- proach achieved the best results on the Se- mEval 2012 Task 2, obtaining a Spearman cor- relation of 0.229 and an accuracy on reproduc- ing human answers to MaxDiff questions of 39.4%. 1 Introduction Considerable prior research has examined and elab- orated upon a wide variety of semantic relations between concepts along with techniques for auto- matically discovering pairs of concepts for which a relation holds (Bejar et al., 1991; Stephens and Chen, 1996; Rosario and Hearst, 2004; Khoo and Na, 2006; Girju et al., 2009). However, most pre- vious work has considered membership assignment for a semantic relation as a binary property. In this paper we discuss an approach which assigns a de- gree of membership to a pair of concepts for a given relation. For example, for the semantic relation CLASS-INCLUSION (Taxonomic), the concept pairs weapon:spear and bird:robin are stronger members Consider the following word pairs: millionaire:money, author:copyright, robin:nest. These X:Y pairs share a relation “X R Y”. Now consider the following word pairs: (1) teacher:students (2) farmer:crops (3) homeowner:door (4) shrubs:roots Which of the numbered word pairs is the MOST illus- trative example of the same relation “X R Y”? Which of the above numbered word pairs is the LEAST illustrative example of the same relation “X R Y”? Figure 1: Example Phase 2 MaxDiff question for the re- lation 2h PART-WHOLE: Creature:Possession. of the relationship than hair:brown, because brown may describe many things other than hair, and brown is also used much less frequently as a noun than the words in the first two word pairs. Task 2 of Se- mEval 2012 (Jurgens et al., 2012) was designed to evaluate the effectiveness of automatic approaches for determining the similarity of a pair of concepts to a specific semantic relation. The task focused on 79 semantic relations from Bejar et al. (1991) which broadly fall into the ten categories enumerated in Ta- ble 1. The data for the task was collected in two phases using Amazon Mechanical Turk 1. During Phase 1, Turkers were asked to provide pairs of words which fit a relation template, such as “X pos- sesses/owns/has Y”. Turkers provided word pairs such as expert:experience, mall:shops, …

Content truncated to 500 words