Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1237 - 1252 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generation Pei Zhou1∗ Karthik Gopalakrishnan2 Behnam Hedayatnia2 Seokhwan Kim2 Jay Pujara1 Xiang Ren1 Yang Liu2 Dilek Hakkani-Tur2 1 Department of Computer Science, University of Southern California 2 Amazon Alexa AI {peiz,jpujara,xiangren}@usc.edu, {karthgop,behnam,seokhwk,yangliud,hakkanit}@amazon.com Abstract Implicit knowledge, such as common sense, is key to fluid human conversations. Current neural response generation (RG) models are trained to generate responses directly, omit- ting unstated implicit knowledge. In this paper, we present Think-Before-Speaking (TBS), a generative approach to first externalize implicit commonsense knowledge (think) and use this knowledge to generate responses (speak). We expect that externalizing implicit knowledge allows more efficient learning, produces more informative responses, and enables more ex- plainable models. We analyze different choices to collect knowledge-aligned dialogues, repre- sent implicit knowledge, and transition between knowledge and dialogues. Empirical results show TBS models outperform end-to-end and knowledge-augmented RG baselines on most automatic metrics and generate more informa- tive, specific, and commonsense-following re- sponses, as evaluated by human annotators. TBS also generates knowledge that makes sense and is relevant to the dialogue around 85% of the time1. 1 Introduction Human communication strives to achieve common ground, consisting of mutual beliefs and common knowledge (Stalnaker, 1978; Clark and Schaefer, 1989). Such common ground depends not only on utterances, but also implicit knowledge. For exam- ple, in Figure 1, this common ground includes the relevant implicit background knowledge “rose is a type of flower”. Integrating such common ground in utterances is an implicit process often referred to as knowledge grounding (Clark and Brennan, 1991). Recent state-of-the-art neural response gen- eration (RG) models based on pre-trained language models (LM) mostly produce responses in an end- to-end manner (Vaswani et al., 2017; Zhang et al., ∗ Work done while Pei Zhou was an intern at Amazon Alexa AI 1Code and data will be released after approval. ��������� ��������� �������� ��������� ���������� ����������������������������������� ������ ��������������������������������� ����� ��������������� �������� ����������������� ������� ��������� �������� ��������� ��������� ���������� ��������������������������������������������� ��������������������������� Figure 1: A motivating example for our study. We look to train models to externalize the implicit knowledge grounding step by explicitly generating knowledge before responding. 2020a; Lewis et al., 2020), i.e., models are trained to take history and produce a response. Since im- plicit knowledge is unstated in dialogue history, RG models do not explicitly learn knowledge ground- ing and may generate uninformative and halluci- nated responses (Serban et al., 2017; Welleck et al., 2019; Roller et al., 2021). Knowledge-grounded RG (Ghazvininejad et al., 2018; Dinan et al., 2019; Gopalakrishnan et al., 2019) addresses this issue, however, most approaches require a knowledge base (KB) to retrieve knowledge for RG (Zhou et al., 2018; Zhao et al., 2020; Eric et al., 2021), which may suffer from the limited knowledge cov- erage of the used KBs. Some work also casts knowledge as a latent …

Content truncated to 500 words