Automatically Building Training Examples for Entity Extraction

Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pages 163–171, Portland, Oregon, USA, 23–24 June 2011. c⃝2011 Association for Computational Linguistics Automatically Building Training Examples for Entity Extraction Marco Pennacchiotti Yahoo! Labs Sunnyvale, CA, USA pennac@yahoo-inc.com Patrick Pantel Microsoft Research Redmond, WA, USA ppantel@microsoft.com Abstract In this paper we present methods for automat- ically acquiring training examples for the task of entity extraction. Experimental evidence show that: (1) our methods compete with a current heavily supervised state-of-the-art sys- tem, within 0.04 absolute mean average pre- cision; and (2) our model significantly out- performs other supervised and unsupervised baselines by between 0.15 and 0.30 in abso- lute mean average precision. 1 Introduction Entity extraction is a fundamental task in NLP and related applications. It is broadly defined as the task of extracting entities of a given semantic class from texts (e.g., lists of actors, musicians, cities). Search engines such as Bing, Yahoo, and Google collect large sets of entities to better interpret queries (Tan and Peng, 2006), to improve query suggestions (Cao et al., 2008) and to understand query intents (Hu et al., 2009). In response, automated techniques for entity extraction have been proposed (Pas¸ca, 2007; Wang and Cohen, 2008; Chaudhuri et al., 2009; Pan- tel et al., 2009). There is mounting evidence that combining knowledge sources and information extraction sys- tems yield significant improvements over applying each in isolation (Pas¸ca et al., 2006; Mirkin et al., 2006). This intuition is explored by the Ensem- ble Semantics (ES) framework proposed by Pennac- chiotti and Pantel (2009), which outperforms pre- vious state-of-the-art systems. A severe limitation of this type of extraction system is its reliance on editorial judgments for building large training sets for each semantic class to be extracted. This is particularly troublesome for applications such as web search that require large numbers of semantic classes in order to have a sufficient coverage of facts and objects (Tan and Peng, 2006). Hand-crafting training sets across international markets is often in- feasible. In an exploratory study we estimated that a pool of editors would need roughly 300 working days to complete a basic set of 100 English classes using the ES framework. Critically needed are meth- ods for automatically building training sets that pre- serve the extraction quality. In this paper, we propose simple and intuitively appealing solutions to automatically build training sets. Positive and negative training sets for a tar- get semantic class are acquired by leveraging: i) ‘trusted’ sources such as structured databases (e.g., IMDB or Wikipedia for acquiring a list of Actors); ii) automatically constructed semantic lexicons; and iii) instances of semantic classes other than the tar- get class. Our models focus on extracting training sets that are large, balanced, and representative of the unlabeled data. These models can be used in any extraction setting, where ‘trusted’ sources of knowl- edge are available: Today, the popularity of struc- tured and semi-structured sources such as Wikipedia and internet databases, makes this approach widely applicable. As an example, in …

Content truncated to 500 words