Data Manipulation: Towards Effective Instance Learning for Neural Dialogue Generation via Learning to Augment and Reweight

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6334–6343 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6334 Data Manipulation: Towards Effective Instance Learning for Neural Dialogue Generation via Learning to Augment and Reweight Hengyi Cai†,§∗, Hongshen Chen‡ Yonghao Song†, Cheng Zhang†, Xiaofang Zhao† and Dawei Yin†† †Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China §University of Chinese Academy of Sciences, Beijing, China ‡Data Science Lab, JD.com, China ††Baidu Inc., China caihengyi@ict.ac.cn, ac@chenhongshen.com, {songyonghao, zhangcheng, zhaoxf}@ict.ac.cn, yindawei@acm.org Abstract Current state-of-the-art neural dialogue mod- els learn from human conversations following the data-driven paradigm. As such, a reliable training corpus is the crux of building a robust and well-behaved dialogue model. However, due to the open-ended nature of human con- versations, the quality of user-generated train- ing data varies greatly, and effective training samples are typically insufficient while noisy samples frequently appear. This impedes the learning of those data-driven neural dialogue models. Therefore, effective dialogue learning requires not only more reliable learning sam- ples, but also fewer noisy samples. In this paper, we propose a data manipulation frame- work to proactively reshape the data distribu- tion towards reliable samples by augmenting and highlighting effective learning samples as well as reducing the effect of inefficient sam- ples simultaneously. In particular, the data manipulation model selectively augments the training samples and assigns an importance weight to each instance to reform the training data. Note that, the proposed data manipula- tion framework is fully data-driven and learn- able. It not only manipulates training samples to optimize the dialogue generation model, but also learns to increase its manipulation skills through gradient descent with validation sam- ples. Extensive experiments show that our framework can improve the dialogue genera- tion performance with respect to various au- tomatic evaluation metrics and human judg- ments. 1 Introduction Open-domain dialogue generation, due to its po- tential applications, is becoming ubiquitous in the community of natural language processing. Cur- rent end-to-end neural dialogue generation mod- els (Li et al., 2016; Serban et al., 2017; Zhao et al., ∗Work done at Data Science Lab, JD.com. inefficient conversations augmentation reweighting effective conversations augmented effective conversations Figure 1: Data manipulation helps the dialogue model training by augmenting and highlighting effective learn- ing samples as well as reducing the weights of ineffi- cient samples. 2017) are primarily built following the data-driven paradigm, that is, these models mimic the human conversations by training on the large-scale query- response pairs. As such, a reliable training corpus that exhibits high-quality conversations is the crux of building a robust and well-behaved dialogue model. Unfortunately, owing to the subjectivity and open-ended nature of human conversations, the quality of the collected human-generated dialogues varies greatly (Shang et al., 2018), which ham- pers the effectiveness of data-driven dialogue mod- els: 1) Effective conversation samples are quite insufficient. To glean some insights on the data quality of dialogue corpus, we choose the query- relatedness to take a glimpse of the data quality. In dialogue corpus, …

Content truncated to 500 words