Iteratively Estimating Pattern Reliability and Seed Quality With Extraction Consistency

Iteratively Estimating Pattern Reliability and Seed Quality With Extraction Consistency * Yi-Hsun Lee a,b, Chung-Yao Chuang b, and Wen-Lian Hsu a,b aInstitue of Information Systems and Applications, National Tsing Hua University, 101, Section 2, Kuang-Fu Road, Hsinchu, Taiwan, R.O.C. bInstitute of Information Science, Academia Sinica, 128 Academia Road, Sec.2, Nankang, Taipei, Taiwan, ROC {rog, cychuang, hsu}@iis.sinica.edu.tw Abstract. In this paper, we focus on the task of distilling relation instances from the Web. Most of the approaches for this task were based on provided seed instances or patterns to initiate the process. Thus, the result of the extraction depends largely on the quality of the instances and patterns. For this matter, we propose an iterative mechanism that estimates the reliability of a pattern by the consistency of its ex- tractions, and reevaluate the usefulness of seed instance based on estimated pattern reliability. The resulting system is a semi-supervised method that can take a large quantity of seed instances with diverse quality. To evaluate the effectiveness of our approach, we experimented on 8 types of relationships. The empirical results show that our system performs quite consistency in different relationships while maintain- ing high precision and recall value. Keywords: consistency estimation, relation extraction, semi-supervised approach, seed quality 1 Introduction The rapid growth of the World Wide Web has attracted a lot of research effort on designing methods that automatically extract knowledge or useful information from large, unstructured text. Different from the conventional corpus, the magnitude and noisy natural of the Web has prohibited analytical approaches to be effective. Consequently, most of the systems that took this challenge proceed in a semi-supervised fashion with a human-provided starting point, such as a few instances of the desired extraction(Mann and Yarowsky, 2005; Muslea, 1999; Ravichandran and Hovy, 2002; Pantel and Pennacchiotti, 2006). In this paper, we focus on the extraction of relation instances. In this scenario, the system needs to be fed with prepared pairs, such as <Barack Obama, Auguest 4th>, or initial extraction patterns to bootstrap. Kozareva and Hovy (2010) mentioned that seed selection plays an important role in this kind of semi-supervised approaches. Therefore, how to select high quality seeds in the initial stage is a critical issue. Most researchers select seeds manually to avoid this problem, but the scalability of such manual selection is not promising. Then, these approaches utilize the given instances, called seeds, and generate extraction patterns that has the potential to locate more instances of the desired type in the text. For example, Ravichandran and Hovy (2002) use surface text patterns like <Person> was born on <Date> to answer questions about birth dates. * This research was supported in part by the National Science Council under grant NSC 100-2631-S-001 -001, and the research center for Humanities and Social Sciences under grant IIS-50-23. Copyright 2011 by Yi-Hsun Lee, Chung-Yao Chuang, and Wen-Lian Hsu 25th Pacific Asia Conference on Language, Information and Computation, pages 382–391 382 {{Infobox NBA Player | name = Kobe Bryant | position = [[Shooting guard]] | team = …

Content truncated to 500 words