GenIE: Generative Information Extraction

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4626 - 4643 July 10-15, 2022 ©2022 Association for Computational Linguistics GenIE: Generative Information Extraction Martin Josifoski,1 Nicola De Cao,2,3 Maxime Peyrard,1 Fabio Petroni,4 Robert West1 1EPFL 2University of Amsterdam, 3University of Edinburgh, 4Meta AI martin.josifoski@epfl.ch, nicola.decao@gmail.com maxime.peyrard@epfl.ch, fabiopetroni@fb.com, robert.west@epfl.ch Abstract Structured and grounded representation of text is typically formalized by closed information extraction, the problem of extracting an exhaus- tive set of (subject, relation, object) triplets that are consistent with a predefined set of entities and relations from a knowledge base schema. Most existing works are pipelines prone to error accumulation, and all approaches are only ap- plicable to unrealistically small numbers of en- tities and relations. We introduce GenIE (gen- erative information extraction), the first end-to- end autoregressive formulation of closed infor- mation extraction. GenIE naturally exploits the language knowledge from the pre-trained trans- former by autoregressively generating relations and entities in textual form. Thanks to a new bi-level constrained generation strategy, only triplets consistent with the predefined knowl- edge base schema are produced. Our experi- ments show that GenIE is state-of-the-art on closed information extraction, generalizes from fewer training data points than baselines, and scales to a previously unmanageable number of entities and relations. With this work, closed information extraction becomes practical in re- alistic scenarios, providing new opportunities for downstream tasks. Finally, this work paves the way towards a unified end-to-end approach to the core tasks of information extraction. 1 Introduction The ability to extract structured semantic informa- tion from unstructured texts is crucial for many AI tasks such as knowledge discovery (Ji and Grish- man, 2011; Trisedya et al., 2019), knowledge main- tenance (Tang et al., 2019), symbolic representa- tion, and reasoning (Ji et al., 2021). The interface between free text and structured knowledge is for- malized by knowledge base population (KBP; Ji and Grishman, 2011), which proposes to represent the information contained in text using (subject, re- lation, object) fact triplets. In this work, we focus on closed information extraction (cIE), the problem US president Biden was born in Pennsylvania. <United States of America, president, Joe Biden> <Joe Biden, ___, ___> Encoder + Autoregressive Decoder Global Structural Constraint Relation Constraint place of death place of birth country death he Entity Constraint California Pennsylvania of the he Figure 1: Overview of GenIE. We use a transformer encoder-decoder model that takes unstructured text as input and autoregressively generates a structured seman- tic representation of the information expressed in it, in the form of (subject, relation, object) triplets. GenIE employs constrained beam search with: (i) a high-level constraint which asserts that the output corresponds to a set of triplets; (ii) lower-level constraints which use prefix tries to force the model to only generate valid entity or relation identifiers (from a predefined schema). of extracting exhaustive sets of fact triplets express- ible under the relation and entity constraints defined by a Knowledge Base (KB) schema. Traditionally, cIE was …

Content truncated to 500 words