Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5382–5391 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5382 Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment Siqi Bao, Huang He, Fan Wang, Rongzhong Lian and Hua Wu Baidu Inc., China {baosiqi, hehuang, wangfan04, lianrongzhong, wu hua}@baidu.com Abstract In this paper, a novel Generation-Evaluation framework is developed for multi-turn conver- sations with the objective of letting both par- ticipants know more about each other. For the sake of rational knowledge utilization and co- herent conversation flow, a dialogue strategy which controls knowledge selection is instan- tiated and continuously adapted via reinforce- ment learning. Under the deployed strategy, knowledge grounded conversations are con- ducted with two dialogue agents. The gener- ated dialogues are comprehensively evaluated on aspects like informativeness and coherence, which are aligned with our objective and hu- man instinct. These assessments are integrated as a compound reward to guide the evolution of dialogue strategy via policy gradient. Com- prehensive experiments have been carried out on the publicly available dataset, demonstrat- ing that the proposed method outperforms the other state-of-the-art approaches significantly. 1 Introduction Intelligent dialogue systems have become popu- lar in our daily life, such as the chit-chat XiaoIce and the task-oriented Echo. These systems serve as smart agents to facilitate more effective inter- action with users in various situations, like ticket booking or recreation offering. Primary dialogue systems (Vinyals and Le, 2015; Shang et al., 2015) try to mimic human beings to generate fluent utter- ances, whereas paying little attention to the intrin- sic factors of human conversations: exchanging information and enhancing interaction (Li et al., 2017). Therefore, they are prone to generate dull and generic responses. To address this problem, in recent years, sev- eral approaches have been developed to generate informative responses based on external knowl- edge. Recently, a knowledge grounded model is proposed in Ghazvininejad et al. (2018), where relevant factual texts are encoded into memory and replies are decoded via attention mechanism. In- stead of using unstructured text knowledge, CCM (Zhou et al., 2018) relies on structured knowledge to generate rich-information response. However, all these approaches are designed for the single- round settings. While applied to the real-world scenarios (where dialogues are conducted for mul- tiple rounds), the dialogue quality will be severely limited due to the lack of coordination among dif- ferent rounds. As discussed above, one of the ultimate goals in human conversation is that information can be exchanged effectively through interaction. Par- ticularly, we argue that successful multi-turn di- alogues are determined by the joint experience of both participants in the conversation, i.e., both participants need to get aware of their counter- parts and express themselves effectively. To this end, we propose the objective of letting both sides know more about each other. With this objective, a novel Generation-Evaluation framework is intro- duced for the multi-turn dialogues. As the name Generation-Evaluation indicates, there are two fundamental modules in our …

Content truncated to 500 words