Toward Diverse Precondition Generation

Proceedings of the 10th Conference on Lexical and Computational Semantics, pages 160–172 August 5–6, 2021, Bangkok, Thailand (online) ©2021 Association for Computational Linguistics 160 Toward Diverse Precondition Generation Heeyoung Kwon1, Nathanael Chambers2, and Niranjan Balasubramanian1 1Stony Brook University, Stony Brook, New York 2 US Naval Academy, Annapolis, MD {heekwon, niranjan}@cs.stonybrook.edu nchamber@usna.edu Abstract Language understanding must identify the log- ical connections between events in a discourse, but core events are often unstated due to their commonsense nature. This paper fills in these missing events by generating precon- dition events. Precondition generation can be framed as a sequence-to-sequence problem: given a target event, generate a possible pre- condition. However, in most real-world scenar- ios, an event can have several preconditions, requiring diverse generation – a challenge for standard seq2seq approaches. We propose DiP, a Diverse Precondition generation system that can generate unique and diverse preconditions. DiP uses a generative process with three com- ponents – an event sampler, a candidate gen- erator, and a post-processor. The event sam- pler provides control codes (precondition trig- gers) which the candidate generator uses to fo- cus its generation. Unlike other conditional generation systems, DiP automatically gener- ates control codes without training on diverse examples. Analysis against baselines reveals that DiP improves the diversity of precondi- tions significantly while also generating more preconditions. 1 Introduction Preconditions are an important part of language un- derstanding with numerous applications, ranging from event understanding to story generation. They provide the semantic glue to understand (or gener- ate) the chains of events common in narrative text. How can we build intelligent systems to fill in these chains, or to identify semantically related events in context? Kwon et al. (2020) took a first step by introducing a precondition generation task, where given a target event mention the goal is to gener- ate text that describes a precondition for the target. They released the ‘PeKo’ dataset for training, and showed that a GPT-2 model can be fine-tuned on TARGET: [BLANK] to fill Mr. Lavelle ’s seat, for a term that expires on Dec. 31, 2008. The Senate voted overwhelmingly on Thursday The Senate voted on Wednesday The Senate voted overwhelmingly on Wednesday The Senate voted overwhelmingly on Tuesday Mr. Lavelle was appointed by Gov. Eliot Spitzer Table 1: Top 5 preconditions generated from GPT-2 with beam search decoding. Key problem: the top 4 preconditions are almost identical. input/output sequence pairs. While PeKo is useful, it is constrained by anno- tating a single relation for each target event. This is contrast to the real-world where most events have many preconditions. For example, “opening a door” has several preconditions like approaching the door, turning a key in the door, and pushing the door. PeKo’s annotation limits the ability of models to learn to generate multiple and diverse preconditions1. In this work, we address the chal- lenge of generating more preconditions for each target event while still maintaining quality. Generating non-repetitive diverse outputs is a challenge for any conditional language generation system. Our analysis of …

Content truncated to 500 words