Accurate Information Extraction from Research Papers using Conditional Random Fields

Accurate Information Extraction from Research Papers using Conditional Random Fields Fuchun Peng Department of Computer Science University of Massachusetts Amherst, MA 01003 fuchun@cs.umass.edu Andrew McCallum Department of Computer Science University of Massachusetts Amherst, MA 01003 mccallum@cs.umass.edu Abstract With the increasing use of research paper search engines, such as CiteSeer, for both lit- erature search and hiring decisions, the accu- racy of such systems is of paramount impor- tance. This paper employs Conditional Ran- dom Fields (CRFs) for the task of extracting various common fields from the headers and citation of research papers. The basic the- ory of CRFs is becoming well-understood, but best-practices for applying them to real-world data requires additional exploration. This paper makes an empirical exploration of several fac- tors, including variations on Gaussian, expo- nential and hyperbolic-L1 priors for improved regularization, and several classes of features and Markov order. On a standard benchmark data set, we achieve new state-of-the-art perfor- mance, reducing error in average F1 by 36%, and word error rate by 78% in comparison with the previous best SVM results. Accuracy com- pares even more favorably against HMMs. 1 Introduction Research paper search engines, such as CiteSeer (Lawrence et al., 1999) and Cora (McCallum et al., 2000), give researchers tremendous power and conve- nience in their research. They are also becoming increas- ingly used for recruiting and hiring decisions. Thus the information quality of such systems is of significant im- portance. This quality critically depends on an informa- tion extraction component that extracts meta-data, such as title, author, institution, etc, from paper headers and references, because these meta-data are further used in many component applications such as field-based search, author analysis, and citation analysis. Previous work in information extraction from research papers has been based on two major machine learn- ing techniques. The first is hidden Markov models (HMM) (Seymore et al., 1999; Takasu, 2003). An HMM learns a generative model over input sequence and labeled sequence pairs. While enjoying wide his- torical success, standard HMM models have difficulty modeling multiple non-independent features of the ob- servation sequence. The second technique is based on discriminatively-trained SVM classifiers (Han et al., 2003). These SVM classifiers can handle many non- independent features. However, for this sequence label- ing problem, Han et al. (2003) work in a two stages pro- cess: first classifying each line independently to assign it label, then adjusting these labels based on an additional classifier that examines larger windows of labels. Solving the information extraction problem in two steps looses the tight interaction between state transitions and obser- vations. In this paper, we present results on this research paper meta-data extraction task using a Conditional Random Field (Lafferty et al., 2001), and explore several practi- cal issues in applying CRFs to information extraction in general. The CRF approach draws together the advan- tages of both finite state HMM and discriminative SVM techniques by allowing use of arbitrary, dependent fea- tures and joint inference over entire sequences. CRFs have been previously applied to other tasks …

Content truncated to 500 words