Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3581–3591 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3581 Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation Wen Zheng University of Nottingham wen.zheng@nottingham.ac.uk Natasa Milic-Frayling University of Nottingham natasa-milic@frayling.net Ke Zhou University of Nottingham & Nokia Bell Labs Ke.Zhou@nottingham.ac.uk Abstract This paper is concerned with improving dia- logue generation models through injection of knowledge, e.g., content relevant to the post that can increase the quality of responses. Past research extends the training of the genera- tive models by incorporating statistical prop- erties of posts, responses and related knowl- edge, without explicitly assessing the knowl- edge quality. In our work, we demonstrate the importance of knowledge relevance and adopt a two-phase approach. We first apply a novel method, Transformer & Post based Posterior Approximation (TPPA) to select knowledge, and then use the Transformer with Expanded Decoder (TED) model to generate responses from both the post and the knowledge. TPPA method processes posts, post related knowl- edge, and response related knowledge at both word and sentence level. Our experiments with the TED generative model demonstrate the effectiveness of TPPA as it outperforms a set of strong baseline models. Our TPPA method is extendable and supports further opti- mization of knowledge retrieval and injection. 1 Introduction In recent years, there have been concerted efforts to model dialogue interactions and generate an appro- priate response to an initial user statement, referred to as a post. Research has led to generative mod- els, e.g., Sequence-to-Sequence (Sutskever et al. (2014)) and Transformer (Vaswani et al., 2017), that produce reasonable responses using the origi- nal post solely during the generation process. Recent studies (Weston et al., 2018; Ghazvinine- jad et al., 2018; Zheng and Zhou, 2019) explored more realistic dialogue models that include knowl- edge related to the posts, typically a collection of sentences that refer to the topics in the posts and responses. Consequently, the response generation Wiz Post: Yep. you’ve got to select for safety standards, of course, but when you’re designing at a Mercedes level the folks buying those cars are going to expect a certain standard of comfort, too! Wiz Response: Especially, I think consumers expect great in Formula One, highest class auto racing. TPPA (top 1): Formula One (also Formula 1 or F1 and officially the FIA Formula One World Championship) is the highest class of single seat auto racing that is sanctioned by the Federation Internationale de l’Automobile (FIA). TPPA (top 2): Stock car racing is a form of automobile racing found mainly and most prominently in the United States and Canada, with Aus- tralia, New Zealand and Brazil also having forms of stock car auto rac- ing. PRK (top 1): Mercedes is part of the McQueen family and is the longest serving McQueen on the series. PRK (top 2): He also won races in midget cars, and sprint cars. RRK (top 1): Formula One (also Formula 1 or F1 and officially the FIA Formula One World …

Content truncated to 500 words