Group-wise Contrastive Learning for Neural Dialogue Generation

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 793–802 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 793 Group-wise Contrastive Learning for Neural Dialogue Generation Hengyi Cai†,§∗, Hongshen Chen‡ Yonghao Song†, Zhuoye Ding‡, Yongjun Bao‡, Weipeng Yan‡, Xiaofang Zhao† †Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China §University of Chinese Academy of Sciences, Beijing, China ‡JD.com, China caihengyi@ict.ac.cn, ac@chenhongshen.com {songyonghao, zhaoxf}@ict.ac.cn, {dingzhuoye, baoyongjun, Paul.yan}@jd.com Abstract Neural dialogue response generation has gained much popularity in recent years. Max- imum Likelihood Estimation (MLE) objective is widely adopted in existing dialogue model learning. However, models trained with MLE objective function are plagued by the low- diversity issue when it comes to the open- domain conversational setting. Inspired by the observation that humans not only learn from the positive signals but also benefit from cor- recting behaviors of undesirable actions, in this work, we introduce contrastive learning into dialogue generation, where the model ex- plicitly perceives the difference between the well-chosen positive and negative utterances. Specifically, we employ a pretrained baseline model as a reference. During contrastive learn- ing, the target dialogue model is trained to give higher conditional probabilities for the posi- tive samples, and lower conditional probabil- ities for those negative samples, compared to the reference model. To manage the multi- mapping relations prevalent in human conver- sation, we augment contrastive dialogue learn- ing with group-wise dual sampling. Exten- sive experimental results show that the pro- posed group-wise contrastive learning frame- work is suited for training a wide range of neu- ral dialogue generation models with very fa- vorable performance over the baseline training approaches. 1 Introduction Open-domain human-machine dialogue systems, especially the generation-based genre, have at- tracted extensive attention recently. Typically, following the neural encoder-decoder paradigm, contemporary dialogue generation models (Shang et al., 2015; Serban et al., 2016; Xing et al., 2017; Yan, 2018; Huang et al., 2020; Liu et al., 2020), ∗Work done at JD.com. more often than not, are trained with Maximum Likelihood Estimation (MLE) principle to mimic human context-response pairs in the training cor- pus. While notable gains have been achieved under this learning framework, prior art (Li et al., 2016a, 2017; Zhang et al., 2018a) suggests that naive MLE objective used for training neural dialogue genera- tion models is not that effective enough and tends to result in issues like dull response generation. By optimizing the likelihood of training dialogues, neural models are inclined to assign high probabili- ties to “safe” responses, due to the fact that vacuous responses like “I don’t know” are of relatively high frequencies in conversational datasets (Li et al., 2016a). One promising training framework for neural dialogue generation is adversarial learning (Good- fellow et al., 2014; Li et al., 2017), where a dis- criminator provides rewards for the generator by contrastively distinguishing dialogues as human- generated or machine-generated. However, the learning ability of GANs in text is drastically limited due to training instability and model col- lapse (Nie et al., 2019; Caccia et al., 2020). First, …

Content truncated to 500 words