Human-like informative conversations: Better acknowledgements using conditional mutual information

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 768–781 June 6–11, 2021. ©2021 Association for Computational Linguistics 768 Human-like informative conversations: Better acknowledgements using conditional mutual information Ashwin Paranjape Stanford University ashwinp@cs.stanford.edu Christopher D. Manning Stanford University manning@cs.stanford.edu Abstract This work aims to build a dialogue agent that can weave new factual content into con- versations as naturally as humans. We draw insights from linguistic principles of conver- sational analysis and annotate human-human conversations from the Switchboard Dialog Act Corpus to examine humans strategies for acknowledgement, transition, detail selection and presentation. When current chatbots (explicitly provided with new factual content) introduce facts into a conversation, their generated responses do not acknowledge the prior turns. This is because models trained with two contexts – new factual content and conversational history – generate responses that are non-specific w.r.t. one of the contexts, typically the conversational history. We show that specificity w.r.t. conversational history is better captured by pointwise conditional mutual information (pcmih) than by the es- tablished use of pointwise mutual information (pmi). Our proposed method, Fused-PCMI, trades off pmi for pcmih and is preferred by humans for overall quality over the Max-PMI baseline 60% of the time. Human evaluators also judge responses with higher pcmih better at acknowledgement 74% of the time. The results demonstrate that systems mimicking human conversational traits (in this case ac- knowledgement) improve overall quality and more broadly illustrate the utility of linguistic principles in improving dialogue agents. 1 Introduction Social chatbots are improving in appeal and are being deployed widely to converse with humans (Gabriel et al., 2020). Advances in neural generation (Adiwardana et al., 2020; Roller et al., 2020) enable them to handle a wide variety of user turns and to provide fluent bot responses. People expect their interactions with these dialogue agents to be similar to real social relationships (Reeves and Nass, 1996). In particular, they expect social Figure 1: The setting for conversational rephrasing. Conversational history (h) and new factual content (k), two largely independent contexts, are used to sample responses (g1, g2) from a generative model. The samples differ qualitatively. While almost all of g2 is verbatim from k (in gray), the first sentence in g1 (in black) acknowledges using h and bridges to k. chatbots to both use information that is already known and separately add new information to the conversation, in line with the given-new contract (Clark and Haviland, 1977). Neural generation methods for adding new infor- mation (Dinan et al., 2019; Gopalakrishnan et al., 2019; Ghazvininejad et al., 2018; Zhang et al., 2018) measure progress using metrics like “engag- ingness”, “appropriateness” and “informativeness”. But these metrics are too broad and provide little actionable insight to drive improvements in these systems. On the other hand, psycholinguists and sociolinguists have studied human conversations in 769 depth and have identified fine-grained conventions, principles and contracts (Grice, 1975; Clark, 1996; Krauss and Fussell, 1996). Our first contribution is a linguistic analysis …

Content truncated to 500 words