Lifelong Event Detection with Knowledge Transfer

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5278–5290 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5278 Lifelong Event Detection with Knowledge Transfer Pengfei Yu1, Heng Ji2, Premkumar Natarajan2 1University of Illinois Urbana-Champaign 2Amazon Alexa AI pengfei4@illinois.edu, {jihj,premknat}@amazon.com Abstract Traditional supervised Information Extrac- tion (IE) methods can extract structured knowl- edge elements from unstructured data, but they are limited to a pre-defined target ontology. In reality, the ontology of interest may change over time, adding emergent new types or more fine-grained subtypes. We propose a new life- long learning framework to address this chal- lenge. We focus on lifelong event detection as an exemplar case and propose a new prob- lem formulation that is also generalizable to other IE tasks. In event detection and more general IE tasks, rich correlations or seman- tic relatedness exist among hierarchical knowl- edge element types. In our proposed frame- work, knowledge is being transferred between learned old event types and new event types. Specifically, we update old knowledge with the mentions of new event types using a self- training loss. In addition, we aggregate the rep- resentations of old event types based on their similarities with new event types to initialize the representations of new event types. Experi- mental results show that our framework outper- forms competitive baselines with a 5.1% abso- lute gain in the F1 score. Moreover, our pro- posed framework can boost the F1 score for over 30% absolute gain on some new long-tail rare event types with few training instances. Our knowledge transfer module improves the performance on both learned event types and new event types under the lifelong learning setting, showing that it helps consolidate old knowledge and improve novel knowledge ac- quisition. 1 1 Introduction The Information Extraction (IE) task aims to ex- tract informative knowledge elements (e.g., enti- ties, relations, and events) from natural language. In practice, we usually extract knowledge elements 1The source code is available at https://github. com/Perfec-Yu/Lifelong-ED. for a pre-defined ontology consisting of various types of knowledge elements of interest. In this setting, IE is often formulated as a classification problem over types in the ontology, with an addi- tional Not-Any (NA) type to identify text spans that don’t belong to any ontology types (negative instances). Ontology-based supervised IE methods such as (Lin et al., 2020) can produce more accurate and structured results with annotated training data than open-domain IE (Yates et al., 2007) while be- ing limited to the ontology. However, the ontology of interest may change over time, adding emergent new types for various knowledge elements (e.g., change bombing to disease outbreak), or adding more fine-grained subtypes for some existing gen- eral types (e.g., break justice into acquit, arrest, charge, convict, release, sentence, and trial). In the past twenty-five years the IE community has been shifting from one old ontology to a new one once every five years based on the consumers’ needs, un- der many shared tasks, including MUC (Grishman and Sundheim, 1996), ACE2, TAC-KBP …

Content truncated to 500 words