Distributed Representations of Words to Guide Bootstrapped Entity Classifiers

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1215–1220, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Distributed Representations of Words to Guide Bootstrapped Entity Classifiers Sonal Gupta Department of Computer Science Stanford University sonal@cs.stanford.edu Christopher D. Manning Department of Computer Science Stanford University manning@stanford.edu Abstract Bootstrapped classifiers iteratively generalize from a few seed examples or prototypes to other examples of target labels. However, sparseness of language and limited supervi- sion make the task difficult. We address this problem by using distributed vector represen- tations of words to aid the generalization. We use the word vectors to expand entity sets used for training classifiers in a bootstrapped pattern-based entity extraction system. Our experiments show that the classifiers trained with the expanded sets perform better on en- tity extraction from four online forums, with 30% F1 improvement on one forum. The re- sults suggest that distributed representations can provide good directions for generalization in a bootstrapping system. 1 Introduction Bootstrapped or distantly-supervised learning is a form of semi-supervised learning, in which supervi- sion is provided by seed examples. Supervised ma- chine learning systems, on the other hand, require hand-labeling sufficient data to train a model, which can be costly and time consuming. Bootstrapped information extraction (IE) has become even more pertinent with the ever-growing amount of data cou- pled with the emergence of open IE systems (Carl- son et al., 2010; Fader et al., 2011) and shared tasks like TAC-KBP.1 Limited supervision provided in bootstrapped systems, though an attractive quality, is also one of 1http://www.nist.gov/tac/2014/KBP its main challenges. When seed sets are small, noisy, or do not cover the label space, the bootstrapped classifiers do not generalize well. We use a major guiding inspiration of deep learn- ing: we can learn a lot about syntactic and semantic similarities between words in an unsupervised fash- ion and capture this information in word vectors. This distributed representation can inform an induc- tive bias to generalize in a bootstrapping system. In this paper, we present a simple approach of us- ing the distributed vector representations of words to expand training data for entity classifiers in a boot- strapped system (see Algorithm 1). To improve the step of learning an entity classifier, we first learn vector representation of entities using the continu- ous bag of words model (Mikolov et al., 2013a). We then use kNN to expand the training set of the clas- sifier by adding unlabeled entities close to seed en- tities in the training set. The key insight is to use the word vector similarity indirectly by enhancing training data for the entity classifier. We do not di- rectly label the unlabeled entities using the similar- ity between word vectors, which we show extracts many noisy entities. We show that classifiers trained with expanded sets of entities perform better on ex- tracting drug-and-treatment entities from four online health forums from MedHelp.2 2 Related Work Bootstrapping has many variants, such as …

Content truncated to 500 words