Domain-Targeted, High Precision Knowledge Extraction

Domain-Targeted, High Precision Knowledge Extraction Bhavana Dalvi Mishra Niket Tandon Allen Institute for Artificial Intelligence 2157 N Northlake Way Suite 110, Seattle, WA 98103 {bhavanad,nikett,peterc}@allenai.org Peter Clark Abstract Our goal is to construct a domain-targeted, high precision knowledge base (KB), contain- ing general (subject,predicate,object) state- ments about the world, in support of a down- stream question-answering (QA) application. Despite recent advances in information extrac- tion (IE) techniques, no suitable resource for our task already exists; existing resources are either too noisy, too named-entity centric, or too incomplete, and typically have not been constructed with a clear scope or purpose. To address these, we have created a domain- targeted, high precision knowledge extraction pipeline, leveraging Open IE, crowdsourcing, and a novel canonical schema learning algo- rithm (called CASI), that produces high pre- cision knowledge targeted to a particular do- main - in our case, elementary science. To measure the KB’s coverage of the target do- main’s knowledge (its “comprehensiveness” with respect to science) we measure recall with respect to an independent corpus of do- main text, and show that our pipeline produces output with over 80% precision and 23% re- call with respect to that target, a substantially higher coverage of tuple-expressible science knowledge than other comparable resources. We have made the KB publicly available1. 1 Introduction While there have been substantial advances in knowledge extraction techniques, the availability of high precision, general knowledge about the world, 1This KB named as “Aristo Tuple KB” is available for down- load at http://data.allenai.org/tuple-kb remains elusive. Specifically, our goal is a large, high precision body of (subject,predicate,object) statements relevant to elementary science, to sup- port a downstream QA application task. Although there are several impressive, existing resources that can contribute to our endeavor, e.g., NELL (Carlson et al., 2010), ConceptNet (Speer and Havasi, 2013), WordNet (Fellbaum, 1998), WebChild (Tandon et al., 2014), Yago (Suchanek et al., 2007), FreeBase (Bollacker et al., 2008), and ReVerb-15M (Fader et al., 2011), their applicability is limited by both • limited coverage of general knowledge (e.g., FreeBase and NELL primarily contain knowl- edge about Named Entities; WordNet uses only a few (< 10) semantic relations) • low precision (e.g., many ConceptNet asser- tions express idiosyncratic rather than general knowledge) Our goal in this work is to create a domain-targeted knowledge extraction pipeline that can overcome these limitations and output a high precision KB of triples relevant to our end task. Our approach leverages existing techniques of open information extraction (Open IE) and crowdsourcing, along with a novel schema learning algorithm. There are three main contributions of this work. First, we present a high precision extraction pipeline able to extract (subject,predicate,object) tuples rele- vant to a domain with precision in excess of 80%. The input to the pipeline is a corpus, a sense- disambiguated domain vocabulary, and a small set of entity types. The pipeline uses a combination of text filtering, Open IE, Turker annotation on sam- ples, and precision prediction to generate its output. 233 Transactions of the Association …

Content truncated to 500 words