Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8554–8565, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8554 Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction Patrick Hohenecker1,2,∗, Frank Mtumbuka1,∗, Vid Kocijan1, Thomas Lukasiewicz1,2 1 University of Oxford, Oxford, UK 2 Serein AI, London, UK firstname.lastname@cs.ox.ac.uk Abstract The goal of open information extraction (OIE) is to extract facts from natural language text, and to represent them as structured triples of the form ⟨subject, predicate, object⟩. For ex- ample, given the sentence »Beethoven com- posed the Ode to Joy.«, we are expected to extract the triple ⟨Beethoven, composed, Ode to Joy⟩. In this work, we systemati- cally compare different neural network archi- tectures and training approaches, and improve the performance of the currently best models on the OIE16 benchmark (Stanovsky and Da- gan, 2016) by 0.421 F1 score and 0.420 AUC- PR, respectively, in our experiments (i.e., by more than 200% in both cases). Furthermore, we show that appropriate problem and loss for- mulations often affect the performance more than the network architecture. 1 Introduction The field of information extraction (IE) focuses on the automatic acquisition of information from text in a structured format (Jurafsky and Martin, 2009; Niklaus et al., 2018), and methods from this field are used to automatically collect desired informa- tion from large corpora of text. Traditionally, IE methods were developed for specific domains with homogeneous corpora and fixed sets of predicates and entities (Niklaus et al., 2018). Such methods are unable to generalize beyond their domains, as they are limited by their predefined collections of entities and predicates. The area of open information extraction (OIE) was introduced as an alternative approach to IE (Banko et al., 2007), where predicates and entities are not restricted to a specific domain. More formally, the aim of an OIE algorithm is to extract all facts en- tailed by the input text in the format of ⟨subject, predicate, object⟩ triples. Alternative formula- tions allow for longer tuples, however, most of the work (including ours) focuses on binary predicates ∗ Equal contribution. only. Given a sentence »Sam succeeded in convinc- ing John.«, an OIE system should extract a tuple: ⟨Sam, succeeded in convincing, John⟩. The relation phrase »succeeded in convincing« indicates the se- mantic relationship between »Sam« and »John«. OIE plays a key role in several downstream natu- ral language processing (NLP) applications, such as knowledge base construction from text (Soder- land et al., 2010), question answering (Fader et al., 2014), information retrieval (Etzioni, 2011), text comprehension, and natural language understand- ing (Mausam, 2016). In this work, we focus on the extraction of binary relations, and introduce several novel neural ap- proaches to OIE. We construct our models from three blocks, namely, an embedding block, an en- coding block, and a prediction block, and introduce several possible implementations of each of them. Furthermore, we exhaustively test all possible com- binations of their use, and show that output encod- ing and loss function strongly …

Content truncated to 500 words