Supervising Unsupervised Open Information Extraction Models

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 728–737, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 728 Supervising Unsupervised Open Information Extraction Models Arpita Roy1, Youngja Park2, Taesung Lee2 and Shimei Pan1 1University of Maryland, Baltimore County Baltimore, MD 21250, USA {arpita2, shimei}@umbc.edu 2 IBM T. J. Watson Research Center Yorktown Heights, New York, USA {young park, taesung.lee}@ibm.com Abstract We propose a novel supervised open infor- mation extraction (Open IE) framework that leverages an ensemble of unsupervised Open IE systems and a small amount of labeled data to improve system performance. It uses the outputs of multiple unsupervised Open IE sys- tems plus a diverse set of lexical and syntactic information such as word embedding, part-of- speech embedding, syntactic role embedding and dependency structure as its input features and produces a sequence of word labels indi- cating whether the word belongs to a relation, the arguments of the relation or irrelevant. Comparing with existing supervised Open IE systems, our approach leverages the knowl- edge in existing unsupervised Open IE sys- tems to overcome the problem of insufficient training data. By employing multiple unsu- pervised Open IE systems, our system learns to combine the strength and avoid the weak- ness in each individual Open IE system. We have conducted experiments on multiple la- beled benchmark data sets. Our evaluation re- sults have demonstrated the superiority of the proposed method over existing supervised and unsupervised models by a significant margin. 1 Introduction Open Information Extraction (Open IE) extracts textual tuples consisting of a relation phrase and argument phrases from a sentence (Banko et al., 2007). Open IE was introduced as an alternative to traditional supervised information extraction (IE) method to address two major limitations of super- vised approaches. First, supervised IE relies heav- ily on labeled training data. Since manual relation annotation is very expensive, this method does not scale to a large number of relations and is very dif- ficult to adapt to new domains. Second, supervised IE systems require the target relations to be prede- termined and learn to extract only the predefined relations. Therefore, they miss new and poten- tially meaningful domain relations that are promi- nent in a given dataset. In contrast, Open IE operates in a completely domain-independent manner and is suitable when the target relations are not known in advance. Re- cently, Open IE has gained much attention, and various Open IE tools have been developed (Fader et al., 2011; Mausam et al., 2012; Akbik and L¨oser, 2012; Corro and Gemulla, 2013; Pal and Mausam, 2016; Yu et al., 2017; Kadry and Dietz, 2017; Roth et al., 2018; Stanovsky et al., 2018). Many Open IE systems demonstrated that they can scale to massive open-domain corpora such as the Web and Wikipedia (Banko et al., 2007), and the extracted tuples can be used as interme- diate representation for various downstream NLP tasks such as knowledge base population (Soder- …

Content truncated to 500 words