OpenFact: Factuality Enhanced Open Knowledge Extraction

OpenFact: Factuality Enhanced Open Knowledge Extraction Linfeng Song∗, Ante Wang∗,†, Xiaoman Pan, Hongming Zhang, Dian Yu, Lifeng Jin, Haitao Mi, Jinsong Su†, Yue Zhang‡ and Dong Yu Tencent AI Lab, Bellevue, WA, USA †School of Informatics, Xiamen University, China ‡School of Engineering, Westlake University, China lfsong@global.tencent.com Abstract We focus on the factuality property during the extraction of an OpenIE corpus named OpenFact, which contains more than 12 mil- lion high-quality knowledge triplets. We break down the factuality property into two im- portant aspects—expressiveness and ground- edness—and we propose a comprehensive framework to handle both aspects. To enhance expressiveness, we formulate each knowledge piece in OpenFact based on a semantic frame. We also design templates, extra constraints, and adopt human efforts so that most Open- Fact triplets contain enough details. For ground- edness, we require the main arguments of each triplet to contain linked Wikidata1 entities. A human evaluation suggests that the OpenFact triplets are much more accurate and contain denser information compared to OPIEC- Linked (Gashteovski et al., 2019), one recent high-quality OpenIE corpus grounded to Wiki- data. Further experiments on knowledge base completion and knowledge base question an- swering show the effectiveness of OpenFact over OPIEC-Linked as supplementary knowl- edge to Wikidata as the major KG. 1 Introduction Open information extraction (OIE, Etzioni et al., 2008) aims to extract factual and informative knowledge, which can further be used to enhance major knowledge bases (Martinez-Rodriguez et al., 2018) or on downstream tasks (Stanovsky et al., 2015). Most current OIE systems (Fader et al., 2011; Mausam et al., 2012; Del Corro and Gemulla, 2013; Angeli et al., 2015) organize knowledge into subject-relation-object (SRO) triples, and they use templates to extract such knowledge triples. ∗Equal contribution, work done during an internship of Ante Wang at Tencent AI Lab. 1https://www.wikidata.org/. Figure 1a and 1b show a Wikipedia2 fragment and the corresponding SRO triple from OPIEC- Linked (Gashteovski et al., 2019), a major OIE corpus. Previous OIE efforts (Fader et al., 2011; Mausam et al., 2012; Del Corro and Gemulla, 2013; Angeli et al., 2015; Stanovsky et al., 2018; Cui et al., 2018) mainly focus on extracting SRO triples that are closer to human annotated out- puts on benchmarks. However, they overlook the factuality issue, where many extracted triples may convey incomplete or even incorrect informa- tion. This issue can be categorized into two main aspects. The first aspect is the lack of expres- siveness, where complex knowledge is poorly or- ganized or critical information is dropped due to the fixed SRO schema and the template-based ex- traction system. Taking Figure 1b as an example, the relation (‘‘surpassed Washington Monument to become’’) is a long span containing multi- ple events, and important temporal information (‘‘during its construction’’) is not captured. The second aspect regards groundedness that measures the degree of a triplet being linked to known en- tities. For instance, a standard OIE system can extract ‘‘<the tower, has, three levels for visi- tors>’’ from input sentence ‘‘The tower has three levels for visitors.’’ …

Content truncated to 500 words