Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4865 - 4877 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations Wei Chen1∗, Yeyun Gong2, Can Xu2, Huang Hu2, Bolun Yao3, Zhongyu Wei1,4, Zhihao Fan1, Xiaowu Hu2, Bartuer Zhou2, Biao Cheng2, Daxin Jiang2 and Nan Duan2 1School of Data Science, Fudan University, China 2Microsoft Research Asia, China 3Nanjing University of Science and Technology, China 4Research Institute of Intelligent and Complex Systems, Fudan University, China {chenwei18,zywei,fanzh18}@fudan.edu.cn;{yegong,caxu,huahu}@microsoft.com, {xiaowuhu, bazhou,bicheng,djiang,nanduan}@microsoft.com; yaobl001@njust.edu.cn Abstract We study the problem of coarse-grained re- sponse selection in retrieval-based dialogue sys- tems. The problem is equally important with fine-grained response selection, but is less ex- plored in existing literature. In this paper, we propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response se- lection in open-domain conversations. In our CFC model, dense representations of query, candidate contexts and responses is learned based on the multi-tower architecture using con- textual matching, and richer knowledge learned from the one-tower architecture (fine-grained) is distilled into the multi-tower architecture (coarse-grained) to enhance the performance of the retriever. To evaluate the performance of the proposed model, we construct two new datasets based on the Reddit comments dump and Twitter corpus. Extensive experimental re- sults on the two datasets show that the proposed method achieves huge improvement over all evaluation metrics compared with traditional baseline methods. 1 Introduction Given utterances of a query, the retrieval-based di- alogue (RBD) system aims to search for the most relevant response from a set of historical records of conversations (Higashinaka et al., 2014; Yan et al., 2016; Boussaha et al., 2019). A complete RBD system usually contain two stages: coarse-grained response selection (RS) and fine-grained response selection (Fu et al., 2020). As shown in Figure 1, in coarse-grained RS stage, the retriever identifies a much smaller list of candidates (usually dozens) from large-scale candidate database (up to millions or more), then the ranker in fine-grained RS stage selects the best response from the retrieved candi- date list. ∗Worked during the internship at Microsoft Research Asia. Zhongyu Wei and Yeyun Gong are corresponding authors. Retriever 𝑄𝑌 𝐶𝑎𝑛𝑑! 𝐶𝑎𝑛𝑑" … 𝐶𝑎𝑛𝑑# Coarse-grained RS Ranker 𝑄𝑌 Fine-grained RS 𝑅𝑒𝑠𝑝𝑜𝑛𝑠𝑒 𝑠𝑜𝑓𝑡𝑚𝑎𝑥(𝑧$) 𝑠𝑜𝑓𝑡𝑚𝑎𝑥(𝑧%) 𝐶𝑎𝑛𝑑! 𝐶𝑎𝑛𝑑" … 𝐶𝑎𝑛𝑑& Fine-to-Coarse Distillation Soft Target Figure 1: A common structure of retrieval-based dia- logue system, where coarse-grained RS provides a much smaller (M ≪ N) candidate set for fine-grained RS. QY and Cand are the abbreviations of query and candi- date respectively. Recent studies (Whang et al., 2020; Xu et al., 2020, 2021; Whang et al., 2021) pay more attention on fine-grained RS and various complex models are proposed to compute the similarities between the query and candidates for response selection. Although promising improvements have been re- ported, the performance of fine-grained stage is inevitably limited by the quality of the candidate list constructed. Therefore, a high-quality coarse- grained RS module is crucial, which is less ex- plored in existing …

Content truncated to 500 words