Not All Seeds Are Equal: Measuring the Quality of Text Mining Seeds

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 618–626, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Not All Seeds Are Equal: Measuring the Quality of Text Mining Seeds Zornitsa Kozareva and Eduard Hovy USC Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292-6695 {kozareva,hovy}@isi.edu Abstract Open-class semantic lexicon induction is of great interest for current knowledge harvest- ing algorithms. We propose a general frame- work that uses patterns in bootstrapping fash- ion to learn open-class semantic lexicons for different kinds of relations. These patterns re- quire seeds. To estimate the goodness (the po- tential yield) of new seeds, we introduce a re- gression model that considers the connectiv- ity behavior of the seed during bootstrapping. The generalized regression model is evaluated on six different kinds of relations with over 10000 different seeds for English and Span- ish patterns. Our approach reaches robust per- formance of 90% correlation coefficient with 15% error rate for any of the patterns when predicting the goodness of seeds. 1 Introduction: What is a Good Seed? The automated construction of semantically typed lexicons (terms classified into their appropriate se- mantic class) from unstructured text is of great im- portance for various kinds of information extraction (Grishman and Sundheim, 1996), question answer- ing (Moldovan et al., 1999), and ontology popu- lation (Suchanek et al., 2007). Maintaining large semantic lexicons is a time-consuming and tedious task, because open classes (such as: all singers, all types of insects) are hard to cover completely, and even closed classes (such as: all countries, all large software companies) change over time. Since it is practically impossible for a human to collect such knowledge adequately, many supervised, unsuper- vised, and semi-supervised techniques have been de- veloped. All these techniques employ some sort of context to specify the appearance in text of the desired in- formation. This approach is based on the general intuition, dating back at least to the distributional similarity idea of (Harris, 1954), that certain con- texts are specific enough to constrain terms or ex- pressions within them to be specific classes or types. Often, the context is a string of words with an empty slot for the desired term(s); sometimes, it is a regu- lar expression-like pattern that includes word classes (syntactic or semantic); sometimes, it is a more ab- stract set of features, including orthographic fea- tures like capitalization, words, syntactic relations, semantic types, and other characteristics, which is the more complete version of the distributional sim- ilarity approach. In early information extraction work, these con- texts were constructed manually, and resembled reg- ular expressions (Appelt et al., 1995). More re- cently, researchers have focused on learning them automatically. Since unsupervised algorithms re- quire large training data and may or may not produce the types and granularities of the semantic class de- sired by the user, and supervised algorithms may re- quire a lot of manual oversight, semi-supervised al- gorithms have become more popular. They require …

Content truncated to 500 words