Event Transition Planning for Open-ended Text Generation

Findings of the Association for Computational Linguistics: ACL 2022, pages 3412 - 3426 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Event Transition Planning for Open-ended Text Generation Qintong Li♥∗ Piji Li♣ Wei Bi♣ Zhaochun Ren♦ Yuxuan Lai♥† Lingpeng Kong♥♠ ♥Department of Computer Science, The University of Hong Kong ♣Tencent AI Lab ♦Shandong University ♠Shanghai Artificial Intelligence Laboratory qtli@connect.hku.hk {lipiji.pz, erutan.pkuicst}@gmail.com victoriabi@tencent.com, zhaochun.ren@sdu.edu.cn lpk@cs.hku.hk Abstract Open-ended text generation tasks, such as dia- logue generation and story completion, require models to generate a coherent continuation given limited preceding context. The open- ended nature of these tasks brings new chal- lenges to the neural auto-regressive text gen- erators nowadays. Despite these neural mod- els are good at producing human-like text, it is difficult for them to arrange causalities and relations between given facts and possible en- suing events. To bridge this gap, we propose a novel two-stage method which explicitly ar- ranges the ensuing events in open-ended text generation. Our approach can be understood as a specially-trained coarse-to-fine algorithm, where an event transition planner provides a “coarse” plot skeleton and a text generator in the second stage refines the skeleton. Experi- ments on two open-ended text generation tasks demonstrate that our proposed method effec- tively improves the quality of the generated text, especially in coherence and diversity. The code is available at: https://github.com/ qtli/EventPlanforTextGen. 1 Introduction With the fast development of large-scale pre-trained models, considerable progress has been made in improving the quality of machine generated text (Radford et al., 2019; Rashkin et al., 2019a; Zhang et al., 2020b; Brown et al., 2020; Guan et al., 2021; Bakhtin et al., 2021). Today, machine learning models can do extremely well in generating text that looks human (Clark et al., 2021). The problem is still far from solved, however, as further read- ing of the machine-generated text often exposes defects such as self-contradiction and topic drift- ing (Bisk et al., 2020; Gao et al., 2020; Tan et al., ∗The majority of this work was done while the first author was interning at Tencent AI Lab. †Corresponding author. Story Context The couples were so happy to be married! The bride and groom were the happiest couple in the world! PG Stage 2 … Event Transition Path XREACT couple feel happy XATTR bride and groom be the happiest couple EP Stage 1 … bride and groom enter OEFFECT the audience cheer … When the bride and groom entered, the audience cheered … ? ? Figure 1: An illustration of our planning based frame- work in story completion task. Given story context, we extract corresponding event transition path, and use model EP to develop potential ensuing event transition paths. The planned paths accordingly guide the path- aware text generation model PG. 2021; Fan et al., 2019; Dou et al., 2021; Dziri et al., 2021). These issues are particularly serious in open- ended text generation tasks (e.g., story completion), where the model is asked to produce a coherent continuation which often involves multiple events, given limited preceding context. …

Content truncated to 500 words