A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2262–2272 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2262 A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation Shilei Liu, Xiaofeng Zhao, Bochao Li, Feiliang Ren∗, Longhui Zhang, Shujuan Yin School of Computer Science and Engineering Key Laboratory of Medical Image Computing of Ministry of Education Northeastern University, Shenyang, 110169, China liusl@live.cn, renfeiliang@cse.neu.edu.cn Abstract Neural conversation models have shown great potentials towards generating fluent and infor- mative responses by introducing external back- ground knowledge. Nevertheless, it is labori- ous to construct such knowledge-grounded di- alogues, and existing models usually perform poorly when transfer to new domains with limited training samples. Therefore, building a knowledge-grounded dialogue system under the low-resource setting is a still crucial is- sue. In this paper, we propose a novel three- stage learning framework based on weakly su- pervised learning which benefits from large scale ungrounded dialogues and unstructured knowledge base. To better cooperate with this framework, we devise a variant of Transformer with decoupled decoder which facilitates the disentangled learning of response generation and knowledge incorporation. Evaluation re- sults on two benchmarks indicate that our ap- proach can outperform other state-of-the-art methods with less training data, and even in zero-resource scenario, our approach still per- forms well. 1 Introduction Neural dialogue systems have made rapid progress in recent years thanks to the advances in sequence generation technology (Vinyals and Le, 2015; Vaswani et al., 2017). Though such models in neural architectures are able to reply with plausi- ble responses regarding to dialogue history, people can still feel a clear gap when they converse with the chatbots, compared with the conversation with humans. To bridge the gap and generate fluent and informative responses, a number of approaches have been proposed by leveraging external knowl- edge. Knowledge-grounded dialogue is a task of generating an informative response based on both dialogue history and a collection of external knowl- edge (Dinan et al., 2019). The forms of knowledge ∗ Corresponding author. are diverse, and in this work, we only focus on knowledge in the form of unstructured documents. Generally, it is difficult to construct large scale conversations that are naturally grounded on the documents for learning of a response generation model (Zhao et al., 2020a), and most of the pre- vious methods (Lian et al., 2019; Li et al., 2019; Kim et al., 2020; Dinan et al., 2019) perform poorly when transfer into a new domain with limited train- ing samples. So there are growing appeals for low- resource dialogue response generation, which aims to leverage past experience to improve the perfor- mance with limited labeled training examples of target corpus. To address this issue, we envisage to absorb use- ful information from other easily accessible het- erogeneous datasets to enhance the performance of the knowledge-based dialogue model under low- resource setting. Based on this assumption, we pro- pose a novel Three-Stage Learning Framework (TSLF). TSLF attempts to divide the parameters of a model into dialogue-related …

Content truncated to 500 words