Open Relation and Event Type Discovery with Type Abstraction

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6864 - 6877 December 7-11, 2022 ©2022 Association for Computational Linguistics Open Relation and Event Type Discovery with Type Abstraction Sha Li, Heng Ji, Jiawei Han University of Illinois Urbana-Champaign {shal2, hengji, hanj}@illinois.edu Abstract Conventional “closed-world" information ex- traction (IE) approaches rely on human ontolo- gies to define the scope for extraction. As a result, such approaches fall short when applied to new domains. This calls for systems that can automatically infer new types from given cor- pora, a task which we refer to as type discovery. To tackle this problem, we introduce the idea of type abstraction, where the model is prompted to generalize and name the type. Then we use the similarity between inferred names to induce clusters. Observing that this abstraction-based representation is often complementary to the entity/trigger token representation, we set up these two representations as two views and de- sign our model as a co-training framework. Our experiments on multiple relation extraction and event extraction datasets consistently show the advantage of our type abstraction approach. 1 Introduction Information extraction has enjoyed widespread suc- cess, however, the majority of information extrac- tion methods are “reactive”, relying on end-users to specify their information needs in prior and pro- vide supervision accordingly. This leads to “closed- world” systems (Lin et al., 2020; Du and Cardie, 2020; Li et al., 2021; Zhong and Chen, 2021; Ye et al., 2022) that are confined to a set of pre-defined types. It is desirable to make systems act more “proactively” like humans who are always on the lookout for interesting new information, generalize them into new types, and find more instances of such types, even if they are not seen previously. One related attempt is the Open Information Ex- traction paradigm (Banko et al., 2008), which aims at extracting all (subject, predicate, object) triples from text that denote some kind of relation. While OpenIE does not rely on pre-specified relations, its exhaustive and free-form nature often leads to noisy and redundant extractions. <h>John</h> earned a bachelor’s degree from the <t>University of Wollongong</t>. Token View University of Wollongong is the [MASK] of John. Mask View Relation: School_Attended Figure 1: For each instance, the token view is computed from the pre-trained LM embedding of the first token in entity/trigger. The mask view is computed from the [MASK] token embedding in the type prompt. To bridge the gap between closed-world IE and OpenIE, a vital step is for systems to possess the ability of automatically inducing new types and extracting instances of such new types. Under vari- ous contexts, related methods have been proposed under the name of “relation discovery” (Yao et al., 2011; Marcheggiani and Titov, 2016),“open rela- tion extraction” (Wu et al., 2019; Hu et al., 2020) and “event type induction” (Huang and Ji, 2020; Shen et al., 2021). In this paper, we unify such terms and refer to the task as type discovery. Type discovery can naturally be posed …

Content truncated to 500 words