PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2513–2525 August 1–6, 2021. ©2021 Association for Computational Linguistics 2513 PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning Siqi Bao∗ Huang He∗ Fan Wang∗ Hua Wu∗ Haifeng Wang Wenquan Wu Zhen Guo Zhibin Liu Xinchao Xu Baidu Inc., China {baosiqi, hehuang, wang.fan, wu hua}@baidu.com Abstract To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained genera- tion model is trained to learn response gener- ation under the simplified framework of one- to-one mapping. In the second stage, a fine- grained generative model augmented with la- tent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and En- glish data, whose effectiveness and superior- ity are verified through comprehensive evalu- ations, achieving new state-of-the-art results. 1 Introduction Recently, task agnostic pre-training with large- scale transformer models has achieved great suc- cess in natural language processing (Devlin et al., 2019), especially open-domain dialogue genera- tion. For instance, based on the general language model GPT-2 (Radford et al., 2019), DialoGPT (Zhang et al., 2020) is further trained for response generation using Reddit comments. To obtain a human-like open-domain chatbot, Meena (Adiwar- dana et al., 2020) scales up the network parameters to 2.6B and employs more social media conver- sations in the training process, leading to signifi- cant improvement on response quality. To mitigate undesirable toxic or bias traits of large corpora, Blender (Roller et al., 2021) fine-tunes the pre- trained model with human annotated datasets and emphasizes desirable conversational skills of en- gagingness, knowledge, empathy and personality. In addition to the attempts from model scale and data selection, PLATO (Bao et al., 2020) aims ∗Equal contribution. Stage 1 Coarse-grained Generation One-to-One Mapping General Response Generation Stage 2.1 Fine-grained Generation One-to-Many Mapping Diverse Response Generation Stage 2.2 Evaluation Response Coherence Estimation Figure 1: Curriculum learning process in PLATO-2. to tackle the inherent one-to-many mapping prob- lem to improve response quality. The one-to-many mapping refers to that one dialogue context might correspond to multiple appropriate responses. It is widely recognized that the capability of model- ing one-to-many relationship is crucial for open- domain dialogue generation (Zhao et al., 2017; Chen et al., 2019). PLATO explicitly models this one-to-many relationship via discrete latent vari- ables, aiming to boost the quality of dialogue gener- ation. PLATO has a modest scale of 132M network parameters and trained with 8M samples, achieving relatively good performance among conversation models on a similar scale. However, scaling up PLATO directly encounters training instability and efficiency issues, which might result from the diffi- culty to capture the one-to-many semantic relation- ship from scratch. In this work, we try to scale up PLATO to PLATO-2 and introduce an effective training schema via curriculum learning (Bengio et al., 2009). There are two stages involved in the …

Content truncated to 500 words