A Synthetic Data Generation Framework for Grounded Dialogues

A Synthetic Data Generation Framework for Grounded Dialogues Jianzhu Bao1,5∗, Rui Wang1,6, Yasheng Wang3, Aixin Sun2, Yitong Li3,4, Fei Mi3, Ruifeng Xu1,5,6† 1Harbin Institute of Technology, Shenzhen, China 2Nanyang Technological University, Singapore 3Huawei Noah’s Ark Lab, 4Huawei Technologies Co., Ltd. 5Peng Cheng Laboratory, Shenzhen, China 6Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies jianzhubao@gmail.com, ruiwangnlp@outlook.com, axsun@ntu.edu.sg, {wangyasheng, mifei2, liyitong3}@huawei.com, xuruifeng@hit.edu.cn Abstract Training grounded response generation models often requires a large collection of grounded di- alogues. However, it is costly to build such dialogues. In this paper, we present a syn- thetic data generation framework (SynDG) for grounded dialogues. The generation pro- cess utilizes large pre-trained language mod- els and freely available knowledge data (e.g., Wikipedia pages, persona profiles, etc.). The key idea of designing SynDG is to consider dialogue flow and coherence in the generation process. Specifically, given knowledge data, we first heuristically determine a dialogue flow, which is a series of knowledge pieces. Then, we employ T5 to incrementally turn the dia- logue flow into a dialogue. To ensure coherence of both the dialogue flow and the synthetic dia- logue, we design a two-level filtering strategy, at the flow-level and the utterance-level respec- tively. Experiments on two public benchmarks show that the synthetic grounded dialogue data produced by our framework is able to signif- icantly boost model performance in both full training data and low-resource scenarios. 1 Introduction Grounded dialogue systems are designed to engage in conversation with humans by incorporating ex- ternal knowledge to provide relevant and informa- tive responses (Ghazvininejad et al., 2018; Dinan et al., 2019; Gopalakrishnan et al., 2019; Zhou et al., 2018b). In recent years, various advanced techniques have been developed to train grounded dialogue models (Zheng et al., 2020; Cui et al., 2021; Xu et al., 2022; Li et al., 2022a). Despite the notable progress, training these models often requires large amounts of data. However, it is ex- pensive and time-consuming to build a collection ∗ This work was partially done during the internship at Huawei Noah’s Ark Lab. †Corresponding Author Husky is a general name for a sled-type of dog used in northern regions, differ- entiated from other sled-dog types by their fast pulling style. Sled dogs were important for transportation in arctic areas, hauling supplies in areas that were inaccessible by other methods. Huskies are also today kept as pets, and groups work to find new pet homes for retired racing and adventure trekking dogs. Dialogue Dialogue Flow I just got a husky puppy. It sounds cute! Huskies are known amongst sled-dogs for their fast pulling style. I guess in the north they are working dogs huh? Sled dogs, including Huskies, are used for transportation in arctic areas. That is so cool and probably helpful but mine is just a pet. That's not uncommon! There are rescue groups that spec- ialize in finding homes for retired sled dogs. Figure 1: An illustrated example from the Wizard of Wikipedia dataset (Dinan et al., 2019). This example shows the dialogue flow in …

Content truncated to 500 words