LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2595–2600 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2595 LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction Jacob Solawetz Roboflow, Inc. Minneapolis, MN, USA jacob@roboflow.ai Stefan Larson Rosegold AI Ann Arbor, MI, USA stefan@rosegold.ai Abstract Open Information Extraction (OIE) systems seek to compress the factual propositions of a sentence into a series of n-ary tuples. These tuples are useful for downstream tasks in natu- ral language processing like knowledge base creation, textual entailment, and natural lan- guage understanding. However, current OIE datasets are limited in both size and diver- sity. We introduce a new dataset by converting the QA-SRL 2.0 dataset to a large-scale OIE dataset (LSOIE). Our LSOIE dataset is 20 times larger than the next largest human-annotated OIE dataset. We construct and evaluate sev- eral benchmark OIE models on LSOIE, provid- ing baselines for future improvements on the task. Our LSOIE data, models, and code are made publicly available.1 1 Introduction Open Information Extraction (OIE) (Banko et al., 2007) aims to automatically extract all factual propositions of a sentence into a series of n-ary tuples. For example, the sentence “the cook baked and ate the cake” would produce two extractions representing the two basic propositions of the sen- tence: (the cook, ate, the cake) and (the cook, baked, the cake). In OIE, extraction arguments are required to be contiguous spans from the sen- tence and the resulting tuple should be intelligible as natural text when read in order. The schema- free nature of OIE provides a flexible framework in which to capture semantic relations between en- tities in natural language text. Open Information Extraction tuples are useful to a variety of down- stream tasks including knowledge base creation (Zhang et al., 2019), textual entailment (Levy et al., 2014), and other natural language understanding tasks (Mausam, 2016). 1Our LSOIE dataset, models, and code can be found at https://github.com/Jacobsolawetz/ large-scale-oie. Domains #Sent. #Ext. OIE2016 Wiki, Newswire 3,180 8,477 AW-OIE Wiki, Wikinews 3,300 17,165 LSOIE-wiki Wiki, Wikinews 24,296 56,662 LSOIE-sci Science 47,998 97,550 Ext. / Sent. Vocab Ordered OIE2016 2.7 13,863 AW-OIE 5.2 15,853 LSOIE-wiki 2.3 46,617 LSOIE-sci 2.0 51,668 Table 1: OIE dataset metrics. Our new dataset LSOIE has substantially more text available than prior work, and includes a new science domain. Our dataset con- version process leverages the scope of the QA-SRL 2.0 bank and improves upon previous methodology. Open Information Extraction relations may be explicitly stated by verbal predicates, or implicitly stated through nominalizations. In this paper, we focus only on explicit extractions. With the orig- inal goal of OIE as web scale information extrac- tion (Banko et al., 2007), an OIE system can focus solely on explicit extractions because the redun- dancy of language will inevitably display implicit information elsewhere. The interest in OIE has grown: both in terms of the types of models that can be applied to tackle OIE (Cui et al., 2018; Stanovsky et …

Content truncated to 500 words