A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4446–4457 August 1–6, 2021. ©2021 Association for Computational Linguistics 4446 A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations Chongyang Tao1∗, Changyu Chen2∗, Jiazhan Feng1, Jirong Wen2,3 and Rui Yan2,3† 1Peking University, Beijing, China 2Gaoling School of Artificial Intelligence, Renmin University of China 3Beijing Academy of Artificial Intelligence 1{chongyangtao,fengjiazhan}@pku.edu.cn 2{chen.changyu,jrwen,ruiyan}@ruc.edu.cn Abstract Recently, many studies are emerging towards building a retrieval-based dialogue system that is able to effectively leverage background knowledge (e.g., documents) when conversing with humans. However, it is non-trivial to collect large-scale dialogues that are naturally grounded on the background documents, which hinders the effective and adequate training of knowledge selection and response matching. To overcome the challenge, we consider decomposing the training of the knowledge-grounded response selection into three tasks including: 1) query-passage matching task; 2) query-dialogue history matching task; 3) multi-turn response matching task, and joint learning all these tasks in a unified pre-trained language model. The former two tasks could help the model in knowledge selection and comprehension, while the last task is designed for matching the proper response with the given query and background knowledge (dialogue history). By this means, the model can be learned to select relevant knowledge and distinguish proper response, with the help of ad-hoc retrieval corpora and a large number of ungrounded multi-turn dialogues. Experimental results on two benchmarks of knowledge-grounded response selection indicate that our model can achieve comparable performance with several existing methods that rely on crowd-sourced data for training. 1 Introduction Along with the very recent prosperity of artificial intelligence empowered conversation systems in the spotlight, many studies have been focused on building human-computer dialogue systems (Wen et al., 2017; Zhang et al., 2020) with either retrieval- based methods (Wang et al., 2013; Wu et al., 2017; ∗Equal Contribution. †Corresponding author: Rui Yan (ruiyan@ruc.edu.cn). Whang et al., 2020) or generation-based meth- ods (Li et al., 2016; Serban et al., 2016; Zhang et al., 2020), which both predict the response with only the given context. In fact, unlike a person who may associate the conversation with the background knowledge in his or her mind, the machine can only capture limited information from the query message itself. As a result, it is difficult for a machine to properly comprehend the query, and to predict a proper response to make it more engaging. To bridge the gap of the knowledge between the human and the machine, researchers have begun to simulating this motivation by grounding dialogue agents with background knowledge (Zhang et al., 2018; Dinan et al., 2019; Li et al., 2020), and lots of impressive results have been obtained. In this paper, we consider the response selection problem in knowledge-grounded conversion and specify the background knowledge as unstructured documents that are common sources in practice. The task is that given a conversation context and a set of knowledge entries, one is required 1): to select proper …

Content truncated to 500 words