A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4927–4939 June 6–11, 2021. ©2021 Association for Computational Linguistics 4927 A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation Yan Zeng DIRO, Université de Montréal yan.zeng@umontreal.ca Jian-Yun Nie DIRO, Université de Montréal nie@iro.umontreal.ca Abstract Conditioned dialogue generation suffers from the scarcity of labeled responses. In this work, we exploit labeled non-dialogue text data re- lated to the condition, which are much eas- ier to collect. We propose a multi-task learn- ing approach to leverage both labeled dialogue and text data. The 3 tasks jointly optimize the same pre-trained Transformer – conditioned dialogue generation task on the labeled di- alogue data, conditioned language encoding task and conditioned language generation task on the labeled text data. Experimental results show that our approach outperforms the state- of-the-art models by leveraging the labeled texts, and it also obtains larger improvement in performance comparing to the previous meth- ods to leverage text data. 1 Introduction General conversational models pre-trained on large text data (Radford et al., 2018; Devlin et al., 2018) or human-to-human conversation data (Zhang et al., 2019; Bao et al., 2019) have shown excellent perfor- mance in generating fluent and diverse responses. In addition to general conversation, we are more and more faced with the problem of conditioned conversation that tunes the dialogue toward a spe- cific style or domain. For example, we might spec- ify a condition as the vocabulary frequently used by a person and ask the system to mimic the speaking style of the person, or a topic-related vocabulary and ask the chatbot to discuss the given topic. Conditioned response generation has been ex- tensively explored using RNN-based sequence-to- sequence models, under different conditions, e.g. persona (Li et al., 2016b), topic (Xing et al., 2017), emotion (Zhou et al., 2018), situations (Sato et al., 2017), and so on. However, only a few existing studies considered using pre-training based models (Zheng et al., 2019; Lin et al., 2019). The basic idea in these previous works is to utilize a paramet- ric vector to represent a condition and then use it in the decoder for conditioned generation. However, the key issue in conditioned dialogue generation is the availability of labeled responses (Zhou and Wang, 2018), and pre-training on unlabeled text or dialogue data does not help much. Therefore, the motivation of this work is to leverage labeled text (non-dialogue) data that are much easier to collect than labeled dialogue data as supplement. These data can be, for example, texts written by the same person (for a persona condition), within the same topic domain (for a topic condition), etc. The idea is inspired by response style transfer (Luan et al., 2017; Niu and Bansal, 2018), which uses a text corpus to learn a style and then transfer the style to dialogue. Based on their success, we assume that the labeled text data can contribute to create better representations of conditions and …

Content truncated to 500 words