Adaptive Parameterization for Neural Dialogue Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1793–1802, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1793 Adaptive Parameterization for Neural Dialogue Generation Hengyi Cai†,§∗, Hongshen Chen‡, Cheng Zhang†, Yonghao Song†, Xiaofang Zhao† and Dawei Yin‡ †Institute of Computing Technology, Chinese Academy of Sciences §University of Chinese Academy of Sciences ‡Data Science Lab, JD.com caihengyi@ict.ac.cn, ac@chenhongshen.com, {zhangcheng, songyonghao, zhaoxf}@ict.ac.cn, yindawei@acm.org Abstract Neural conversation systems generate re- sponses based on the sequence-to-sequence (SEQ2SEQ) paradigm. Typically, the model is equipped with a single set of learned pa- rameters to generate responses for given in- put contexts. When confronting diverse con- versations, its adaptability is rather limited and the model is hence prone to generate generic responses. In this work, we propose an Adaptive Neural Dialogue generation model, ADAND, which manages various conversa- tions with conversation-specific parameteriza- tion. For each conversation, the model gener- ates parameters of the encoder-decoder by re- ferring to the input context. In particular, we propose two adaptive parameterization mech- anisms: a context-aware and a topic-aware pa- rameterization mechanism. The context-aware parameterization directly generates the param- eters by capturing local semantics of the given context. The topic-aware parameterization en- ables parameter sharing among conversations with similar topics by first inferring the latent topics of the given context and then generat- ing the parameters with respect to the distri- butional topics. Extensive experiments con- ducted on a large-scale real-world conversa- tional dataset show that our model achieves su- perior performance in terms of both quantita- tive metrics and human evaluations. 1 Introduction Open-domain dialogue models, usually called chit- chat systems, draw increasing attention from both academia and industry in recent years. Building on the successful sequence-to-sequence (SEQ2SEQ) paradigm (Sutskever et al., 2014; Cho et al., 2014; Bahdanau et al., 2015), contemporary mainstream open-domain dialogue generation models (Shang et al., 2015; Serban et al., 2016, 2017; Shen et al., 2017; Clark and Cao, 2017; Zhao et al., 2017; Xing ∗Work done at Data Science Lab, JD.com. et al., 2017; Zhao et al., 2018), trained on a large number of context-response pairs, attempt to gen- erate an appropriate response for the given context based on a single set of the model parameters. Because of its great potential in understanding and modeling conversations, SEQ2SEQ has been widely applied in different kinds of conversation scenarios including technical supporting, movie discussions, and social entertainment, etc. How- ever, when confronting conversations with diverse topics or themes, SEQ2SEQ is usually prone to make generic meaningless responses due to its over- simplified parameterization. To tackle this issue, Xing et al. (2017) proposed a topic-aware response generation model, which utilizes a single encoder- decoder, augmented with topic information ob- tained from a pre-trained LDA model. Though effective, the model heavily relies on the outsourc- ing topic information to capture the topic varia- tions of different conversations. Another approach, per-topic/theme encoder-decoder model (Choud- hary et al., 2017), uses separate encoder-decoder model …

Content truncated to 500 words