ProphetChat: Enhancing Dialogue Generation with Simulation of Future Conversation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 962 - 973 May 22-27, 2022 c⃝2022 Association for Computational Linguistics ProphetChat: Enhancing Dialogue Generation with Simulation of Future Conversation Chang Liu1,2, Xu Tan3, Chongyang Tao4, Zhenxin Fu1, Dongyan Zhao1,2,5,6∗, Tie-Yan Liu3, Rui Yan7∗ 1Wangxuan Institute of Computer Technology, Peking University 2Center for Data Science, Peking University 3Microsoft Research Asia 4Microsoft Corporation 5Artificial Intelligence Institute of Peking University 6State Key Laboratory of Media Convergence Production Technology and Systems 7Gaoling School of Artificial Intelligence, Renmin University of China {liuchang97,fuzhenxin,zhaody}@pku.edu.cn, {xuta,chotao,tyliu}@microsoft.com ruiyan@ruc.edu.cn Abstract Typical generative dialogue models utilize the dialogue history to generate the response. How- ever, since one dialogue utterance can often be appropriately answered by multiple dis- tinct responses, generating a desired response solely based on the historical information is not easy. Intuitively, if the chatbot can fore- see in advance what the user would talk about (i.e., the dialogue future) after receiving its response, it could possibly provide a more informative response. Accordingly, we pro- pose a novel dialogue generation framework named ProphetChat that utilizes the simulated dialogue futures in the inference phase to en- hance response generation. To enable the chat- bot to foresee the dialogue future, we design a beam-search-like roll-out strategy for dialogue future simulation using a typical dialogue gen- eration model and a dialogue selector. With the simulated futures, we then utilize the en- semble of a history-to-response generator and a future-to-response generator to jointly gener- ate a more informative response. Experiments on two popular open-domain dialogue datasets demonstrate that ProphetChat can generate bet- ter responses over strong baselines, which vali- dates the advantages of incorporating the simu- lated dialogue futures. 1 Introduction Recent years have witnessed a surge of interest in building open-domain chatbots using generative approaches (Shang et al., 2015; Zhao et al., 2017; Tao et al., 2018; Zhang et al., 2020). These prevail- ing methods typically utilize dialogue histories as the dialogue context to generate the response via maximum likelihood estimation. Different from ∗Corresponding authors: Dongyan Zhao and Rui Yan. Figure 1: A generation case of ProphetChat. It first sim- ulates the dialogue future then generates the response conditioned on both the history and the future. directed text generation tasks like machine trans- lation where the target sentences are strictly con- strained by the source sentence (Holtzman et al., 2019), the dialogue history and the response in chit- chat conversations are loosely coupled (Feng et al., 2020a). In other words, open-domain chatbots of- ten have more “freedom" to decide what to respond since there often exists multiple distinct responses that can appropriately answer the given utterance. However, we argue that such excessive “freedom" also reveals that the dialogue history only may not contain enough information to generate a desired response that is informative and easy to reply to. If provided with enriched dialogue contexts that contain more useful dialogue cues, it could be eas- ier for the model to chat with a human. So here …

Content truncated to 500 words