Integrating Probabilistic Extraction Models and Data Mining to Discover Relations and Patterns in Text

Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL, pages 296–303, New York, June 2006. c⃝2006 Association for Computational Linguistics Integrating Probabilistic Extraction Models and Data Mining to Discover Relations and Patterns in Text Aron Culotta University of Massachusetts Amherst, MA 01003 culotta@cs.umass.edu Andrew McCallum University of Massachusetts Amherst, MA 01003 mccallum@cs.umass.edu Jonathan Betz Google, Inc. New York, NY 10018 jtb@google.com Abstract In order for relation extraction systems to obtain human-level performance, they must be able to incorporate relational pat- terns inherent in the data (for example, that one’s sister is likely one’s mother’s daughter, or that children are likely to attend the same college as their par- ents). Hand-coding such knowledge can be time-consuming and inadequate. Addi- tionally, there may exist many interesting, unknown relational patterns that both im- prove extraction performance and provide insight into text. We describe a probabilis- tic extraction model that provides mutual benefits to both “top-down” relational pat- tern discovery and “bottom-up” relation extraction. 1 Introduction Consider these four sentences: 1. George W. Bush’s father is George H. W. Bush. 2. George H. W. Bush’s sister is Nancy Bush Ellis. 3. Nancy Bush Ellis’s son is John Prescott Ellis. 4. John Prescott Ellis analyzed George W. Bush’s campaign. We would like to build an automated system to extract the set of relations shown in Figure 1. cousin Nancy Ellis Bush sibling George HW Bush George W Bush son John Prescott Ellis son Figure 1: Bush family tree State of the art extraction algorithms may be able to detect the son and sibling relations from local lan- guage clues. However, the cousin relation is only implied by the text and requires additional knowl- edge to be extracted. Specifically, the system re- quires knowledge of familial relation patterns. One could imagine a system that accepts such rules as input (e.g. cousin = father’s sister’s son) and applies them to extract implicit relations. How- ever, exhaustively enumerating all possible rules can be tedious and incomplete. More importantly, many relational patterns unknown a priori may both im- prove extraction accuracy and uncover informative trends in the data (e.g. that children often adopt the religion of their parents). Indeed, the goal of data mining is to learn such patterns from database reg- ularities. Since these patterns will not always hold, we would like to handle them probabilistically. We propose an integrated supervised machine learning method that learns both contextual and re- lational patterns to extract relations. In particular, we construct a linear-chain conditional random field (Lafferty et al., 2001; Sutton and McCallum, 2006) to extract relations from biographical texts while si- multaneously discovering interesting relational pat- terns that improve extraction performance. 296 2 Related Work This work can be viewed as a step toward the in- tegration of information extraction and data mining technology, a direction of growing interest. Nahm and Mooney (2000) present a system that mines as- sociation rules from a database constructed from au- tomatically extracted data, then applies these …

Content truncated to 500 words