Semi-Automatic Entity Set Refinement

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 290–298, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Semi-Automatic Entity Set Refinement Vishnu Vyas and Patrick Pantel Yahoo! Labs Santa Clara, CA 95054 {vishnu,ppantel}@yahoo-inc.com Abstract State of the art set expansion algorithms pro- duce varying quality expansions for different entity types. Even for the highest quality ex- pansions, errors still occur and manual re- finements are necessary for most practical uses. In this paper, we propose algorithms to aide this refinement process, greatly reducing the amount of manual labor required. The me- thods rely on the fact that most expansion er- rors are systematic, often stemming from the fact that some seed elements are ambiguous. Using our methods, empirical evidence shows that average R-precision over random entity sets improves by 26% to 51% when given from 5 to 10 manually tagged errors. Both proposed refinement models have linear time complexity in set size allowing for practical online use in set expansion systems. 1 Introduction Sets of named entities are extremely useful in a variety of natural language and information re- trieval tasks. For example, companies such as Ya- hoo! and Google maintain sets of named entities such as cities, products and celebrities to improve search engine relevance. Manually creating and maintaining large sets of named entities is expensive and laborious. In re- sponse, many automatic and semi-automatic me- thods of creating sets of named entities have been proposed, some are supervised (Zhou and Su, 2001), unsupervised (Pantel and Lin 2002, Nadeau et al. 2006), and others semi-supervised (Kozareva et al. 2008). Semi-supervised approaches are often used in practice since they allow for targeting spe- cific entity classes such as European Cities and French Impressionist Painters. Methods differ in complexity from simple ones using lexico- syntactic patterns (Hearst 1992) to more compli- cated techniques based on distributional similarity (Paşca 2007a). Even for state of the art methods, expansion er- rors inevitably occur and manual refinements are necessary for most practical uses requiring high precision (such as for query interpretation at com- mercial search engines). Looking at expansions from state of the art systems such as GoogleSets1 , we found systematic errors such as those resulting from ambiguous seed instances. For example, con- sider the following seed instances for the target set Roman Gods: Minerva, Neptune, Baccus, Juno, Apollo GoogleSet’s expansion as well others employing distributional expansion techniques consists of a mishmash of Roman Gods and celestial bodies, originating most likely from the fact that Neptune is both a Roman God and a Planet. Below is an excerpt of the GoogleSet expansion: Mars, Venus, *Moon, Mercury, *asteroid, Jupiter, *comet, *Sonne, *Sun, … *Earth, The inherent semantic similarity between the errors can be leveraged to quickly clean up the expan- sion. For example, given a manually tagged error “asteroid”, a distributional similarity thesaurus 1 http://labs.google.com/sets 290 such as (Lin 1998)2 can identify comet as similar to asteroid and therefore potentially also as an error. This …

Content truncated to 500 words