Template Filling with Generative Transformers

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 909–914 June 6–11, 2021. ©2021 Association for Computational Linguistics 909 Template Filling with Generative Transformers Xinya Du Alexander M. Rush Department of Computer ScienceClaire Cardie Cornell University {xdu, cardie}@cs.cornell.edu arush@cornell.edu Abstract Template filling is generally tackled by a pipeline of two separate supervised systems – one for role-filler extraction and another for template/event recognition. Since pipelines consider events in isolation, they can suffer from error propagation. We introduce a frame- work based on end-to-end generative trans- formers for this task (i.e., GTT). It naturally models the dependence between entities both within a single event and across the multiple events described in a document. Experiments demonstrate that this framework substantially outperforms pipeline-based approaches, and other neural end-to-end baselines that do not model between-event dependencies. We fur- ther show that our framework specifically im- proves performance on documents containing multiple events. 1 Introduction The classic template-filling task in information ex- traction involves extracting event-based templates from documents (Grishman and Sundheim, 1996; Jurafsky and Martin, 2009; Grishman, 2019). It is usually tackled by a pipeline of two separate sys- tems, one for role-filler entity extraction – extract- ing event-relevant entities (e.g., noun phrases) from the document; another for template/event recogni- tion – assigning each of the candidate role-fillers to the event(s)/template(s) that it participates in and identifying the type of each event/template. Simplifications of the task (Patwardhan and Riloff, 2009; Huang and Riloff, 2011, 2012; Du et al., 2020) assume that there is one generic tem- plate and focus only on role-filler entity extraction. However, real documents often describe multiple events (Figure 1). From the example, we can ob- serve that between-event dependencies are impor- tant (e.g., a single organization can participate in multiple events) and can span the entire document (e.g., event-specific targets can be distant from their Several attacks were carried out in La Paz last night, one in front of government house ... The self-styled "Zarate armed forces" sent simultaneous written messages to the media, calling on the people to oppose ... The first attack occurred at 22:30 in front of the economic ministry, just before President Paz Zamora concluded his message to ... Roberto Barbery, has reported that dynamite sticks were hurled from a car. The second attack occurred at 23:35, just after the cabinet members had left government house where they had listened to the presidential message. A bomb was placed outside government house in the parking lot that is used by cabinet ministers. The police ... As of 5:00 today, people found that an old shack on the estate was set ablaze, Event 2 Template Bombing Perpetrator Indiv. - Perpetrator Org Zarate armed forces Physical Target government house Weapon bomb Victim - Event 1 Template Attack Perpetrator Indiv. - Perpetrator Org Zarate armed forces Physical Target economic ministry Weapon dynamite sticks Victim - Event 3 Template Arson Perpetrator Indiv. - Perpetrator Org Zarate armed forces Physical Target …

Content truncated to 500 words