Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7382–7394 July 9-14, 2023 ©2023 Association for Computational Linguistics Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation Ang Lv1,∗, Jinpeng Li2∗, Shufang Xie1, Rui Yan1,3† 1Gaoling School of Artifical Intelligence, Renmin University of China 2Wangxuan Institute of Computer Technology, Peking University 3Engineering Research Center of Next-Generation Intelligent Search and Recommendation, Ministry of Education {anglv, shufangxie, ruiyan}@ruc.edu.cn, lijinpeng@stu.pku.edu.cn Abstract In this paper, we define a widely neglected prop- erty in dialogue text, duality, which is a hier- archical property that is reflected in human be- haviours in daily conversations: Based on the logic in a conversation (or a sentence), people can infer follow-up utterances (or tokens) based on the previous text, and vice versa. We pro- pose a hierarchical duality learning for dialogue (HDLD) to simulate this human cognitive abil- ity, for generating high quality responses that connect both previous and follow-up dialogues. HDLD utilizes hierarchical dualities at token hierarchy and utterance hierarchy. HDLD maxi- mizes the mutual information between past and future utterances. Thus, even if the future text is invisible during inference, HDLD is capa- ble of estimating future information implicitly based on dialogue history and generates both coherent and informative responses. In con- trast to previous approaches that solely utilize future text as auxiliary information to encode during training, HDLD leverages duality to en- able interaction between dialogue history and the future. This enhances the utilization of dia- logue data, leading to the improvement in both automatic and human evaluation. 1 Introduction Dialogue generation achieves great success in re- cent years. In this task, a dialogue session lasts for multiple turns, and the goal is to predict the response to the previous context. Researchers have explored various aspects of dialogue generation models including context-aware (Tian et al., 2017), persona-based (Qian et al., 2018) and knowledge- grounded (Zhao et al., 2020; Tao et al., 2021), etc. Because dialogue generation is a context-aware process, there is a large body of literature on the context modeling among all research lines. In the pilot studies, contexts are modeled as the concatenation of previous utterances (Sordoni et al., ∗Equal contribution. †Corresponding author: Rui Yan (ruiyan@ruc.edu.cn). Are you traveling around Europe? Great! Make sure you buy me a souvenir! Oh, don’t worry. I will! Yes, I'm shopping in a scenic spot in Paris. (a) Inter-duality. I rent a new house because I had an argument with roommates. (b) Intra-duality. Figure 1: (a) Inter-duality: the dialogue is divided into two parts. According to the first two utterances, we can infer that the follow-up part is related to the scenic spot commodities. Similarly, based on the last two utterances, we can infer that the previous conversation is about tourism. (b) Intra-duality: in an utterance, given the blue part, people know that the follow-up part is the reason. Given the pink part, people know that the previous tokens may explain the result of the argument. 2015), …

Content truncated to 500 words