Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3426–3437, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3426 Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation Xiuyi Chen1,2,3, Fandong Meng4, Peng Li4, Feilong Chen1,2,3, Shuang Xu1, Bo Xu1,2,3, Jie Zhou4 1Institute of Automation, Chinese Academy of Sciences (CASIA). Beijing, China 2Research Center for Brain-inspired Intelligence, CASIA 3University of Chinese Academy of Sciences, Beijing, China {chenxiuyi2017,chenfeilong2018,shuang.xu,xubo}@ia.ac.cn 4Pattern Recognition Center, WeChat AI, Tencent Inc, Beijing, China {fandongmeng,patrickpli,withtomzhou}@tencent.com Abstract Knowledge selection plays an important role in knowledge-grounded dialogue, which is a challenging task to generate more informative responses by leveraging external knowledge. Recently, latent variable models have been pro- posed to deal with the diversity of knowledge selection by using both prior and posterior dis- tributions over knowledge and achieve promis- ing performance. However, these models suf- fer from a huge gap between prior and poste- rior knowledge selection. Firstly, the prior se- lection module may not learn to select knowl- edge properly because of lacking the necessary posterior information. Secondly, latent vari- able models suffer from the exposure bias that dialogue generation is based on the knowledge selected from the posterior distribution at train- ing but from the prior distribution at inference. Here, we deal with these issues on two as- pects: (1) We enhance the prior selection mod- ule with the necessary posterior information obtained from the specially designed Posterior Information Prediction Module (PIPM); (2) We propose a Knowledge Distillation Based Training Strategy (KDBTS) to train the de- coder with the knowledge selected from the prior distribution, removing the exposure bias of knowledge selection. Experimental results on two knowledge-grounded dialogue datasets show that both PIPM and KDBTS achieve per- formance improvement over the state-of-the- art latent variable model and their combination shows further improvement. 1 Introduction Knowledge-grounded dialogue (Ghazvininejad et al., 2018) which leverages external knowledge to generate more informative responses, has become a popular research topic in recent years. Many researchers have studied how to effectively lever- age the given knowledge to enhance dialogue un- derstanding and/or improve dialogue generation Context I just got a husky puppy. Knowledge Pool # Knowledge Sentence 0 Husky is a general name for a sled type of dog used in northern regions, differentiated from other sled-dog types by their fast pulling style. 1 Huskies are also today kept as pets, and groups work to find new pet homes for retired racing and adventure trekking dogs. 2 Huskies are used in sled dog racing. 3 The use of “husk” is recorded from 1852 for dogs kept by Inuit people ... ... L Child of the Wolves is a children’s novel, published in 1996, about a Siberian husky puppy that joins a wolf pack. Response a It sounds cute! Huskies are known amongst sled-dogs for their fast pulling style. Response b It sounds cute! I have read a novel about a husky puppy joining a wolf pack. Is your husky puppy wolf-like in appearance? Table 1: …

Content truncated to 500 words