Unsupervised Learning of Field Segmentation Models for Information Extraction

Proceedings of the 43rd Annual Meeting of the ACL, pages 371–378, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Unsupervised Learning of Field Segmentation Models for Information Extraction Trond Grenager Computer Science Department Stanford University Stanford, CA 94305 grenager@cs.stanford.edu Dan Klein Computer Science Division U.C. Berkeley Berkeley, CA 94709 klein@cs.berkeley.edu Christopher D. Manning Computer Science Department Stanford University Stanford, CA 94305 manning@cs.stanford.edu Abstract The applicability of many current information ex- traction techniques is severely limited by the need for supervised training data. We demonstrate that for certain field structured extraction tasks, such as classified advertisements and bibliographic ci- tations, small amounts of prior knowledge can be used to learn effective models in a primarily unsu- pervised fashion. Although hidden Markov models (HMMs) provide a suitable generative model for field structured text, general unsupervised HMM learning fails to learn useful structure in either of our domains. However, one can dramatically im- prove the quality of the learned structure by ex- ploiting simple prior knowledge of the desired so- lutions. In both domains, we found that unsuper- vised methods can attain accuracies with 400 un- labeled examples comparable to those attained by supervised methods on 50 labeled examples, and that semi-supervised methods can make good use of small amounts of labeled data. 1 Introduction Information extraction is potentially one of the most useful applications enabled by current natural lan- guage processing technology. However, unlike gen- eral tools like parsers or taggers, which generalize reasonably beyond their training domains, extraction systems must be entirely retrained for each appli- cation. As an example, consider the task of turn- ing a set of diverse classified advertisements into a queryable database; each type of ad would require tailored training data for a supervised system. Ap- proaches which required little or no training data would therefore provide substantial resource savings and extend the practicality of extraction systems. The term information extraction was introduced in the MUC evaluations for the task of finding short pieces of relevant information within a broader text that is mainly irrelevant, and returning it in a struc- tured form. For such “nugget extraction” tasks, the use of unsupervised learning methods is difficult and unlikely to be fully successful, in part because the nuggets of interest are determined only extrinsically by the needs of the user or task. However, the term information extraction was in time generalized to a related task that we distinguish as field segmenta- tion. In this task, a document is regarded as a se- quence of pertinent fields, and the goal is to segment the document into fields, and to label the fields. For example, bibliographic citations, such as the one in Figure 1(a), exhibit clear field structure, with fields such as author, title, and date. Classified advertise- ments, such as the one in Figure 1(b), also exhibit field structure, if less rigidly: an ad consists of de- scriptions of attributes of an item or offer, and a set of ads for similar items share the same attributes. In …

Content truncated to 500 words