SubeventWriter: Iterative Sub-event Sequence Generation with Coherence Controller

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1590 - 1604 December 7-11, 2022 ©2022 Association for Computational Linguistics SubeventWriter: Iterative Sub-event Sequence Generation with Coherence Controller Zhaowei Wang1, Hongming Zhang2, Tianqing Fang1, Yangqiu Song1, Ginny Y. Wong3, & Simon See3 1Department of Computer Science and Engineering, HKUST 2Tencent AI Lab, Bellevue, USA 3NVIDIA AI Technology Center (NVAITC), NVIDIA, Santa Clara, USA {zwanggy, tfangaa, yqsong}@cse.ust.hk hongmzhang@global.tencent.com, {gwong, ssee}@nvidia.com Abstract In this paper, we propose a new task of sub- event generation for an unseen process to eval- uate the understanding of the coherence of sub- event actions and objects. To solve the prob- lem, we design SubeventWriter, a sub-event sequence generation framework with a coher- ence controller. Given an unseen process, the framework can iteratively construct the sub- event sequence by generating one sub-event at each iteration. We also design a very effective coherence controller to decode more coherent sub-events. As our extensive experiments and analysis indicate, SubeventWriter1 can gener- ate more reliable and meaningful sub-event se- quences for unseen processes. 1 Introduction Natural language understanding involves deep un- derstanding of events. In the NLP community, there have been many event understanding tasks. Most of them focus on parsing events into in- volved entities, time, and locations as semantic roles (Kingsbury and Palmer, 2002; Li et al., 2013; Lv et al., 2020; Lin et al., 2020; Du and Cardie, 2020; Zhang et al., 2021; Lyu et al., 2021a), or identifying their binary relations such as tempo- ral or causal relations (Berant et al., 2014; Smith et al., 2018; Sap et al., 2019; Wang et al., 2020, 2021). However, our natural language can be used to describe relations more than binary ones. For ex- ample, processes (Craig et al., 1998), also known as scripts (Schank and Abelson, 1977) or activi- ties (Mourelatos, 1978), are complex events consti- tuted by a sequence of sub-events. Understanding processes can be more challenging than individual or pair of events. As shown in Figure 1, to complete the process of making a chocolate cake, we need to consider a se- quence of actions, “mix,” “add,” “pour,” and “bake,” 1Code is available at https://github.com/ HKUST-KnowComp/SubeventWriter. Process: make a chocolate cake How to make a chocolate cake? Step 1: mix the dry ingredients. Step 2: [M] Generate the 2nd sub-event Beam Search Candidates Problem Coherent? Add the carrots Add the coffee and milk Mix the dry ingredients Right one Redundant Irrelevant Sub-event Reference 1. Mix the dry ingredients. 3. Pour batter into cake pans. 2. Add the coffee and milk. 4. Bake the cake. SubeventWriter Figure 1: A motivating example of SubeventWriter, which generates one sub-event at a time iteratively. We show the process “make a chocolate cake” and the sec- ond iteration of the generation. By considering coher- ence, we can re-rank candidates and reach the right sub-event. [M] is a mask token. which involves different objects, e.g., dry ingredi- ents, coffee, milk, etc. Those actions should follow a logically coherent procedure …

Content truncated to 500 words