AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1183–1195 August 1–6, 2021. ©2021 Association for Computational Linguistics 1183 AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation Xinnuo Xu1, Guoyin Wang2, Young-Bum Kim2, Sungjin Lee2 1The Interaction Lab, Heriot-Watt University, Edinburgh 2Amazon Alexa AI, Seattle, WA, USA xx6@hw.ac.uk, guoyiwan, youngbum, sungjinl@amazon.com Abstract Natural Language Generation (NLG) is a key component in a task-oriented dialogue system, which converts the structured meaning repre- sentation (MR) to the natural language. For large-scale conversational systems, where it is common to have over hundreds of intents and thousands of slots, neither template-based ap- proaches nor model-based approaches are scal- able. Recently, neural NLGs started lever- aging transfer learning and showed promis- ing results in few-shot settings. This paper proposes AUGNLG, a novel data augmenta- tion approach that combines a self-trained neu- ral retrieval model with a few-shot learned NLU model, to automatically create MR-to- Text data from open-domain texts. The pro- posed system mostly outperforms the state-of- the-art methods on the FEWSHOTWOZ data in both BLEU and Slot Error Rate. We fur- ther confirm improved results on the FEW- SHOTSGD data and provide comprehensive analysis results on key components of our sys- tem. Our code and data are available at https: //github.com/XinnuoXu/AugNLG. 1 Introduction Large-scale conversational systems provide a nat- ural interface to achieve various daily-life tasks. Natural Language Generation (NLG) is a key com- ponent in such a system to convert the structured meaning representation (MR) to the natural lan- guage, as shown in Figure 1. In task-oriented dia- logue systems, NLG is typically accomplished by filling out a basic set of developer-provided tem- plates, leading to a conversational system generat- ing unnatural, robotic responses. In order to make the system sound more human-like, model-based NLG approaches, in particular neural models, have recently been gaining an increasing traction (Gao et al., 2018; Wen et al., 2015). However, neither the template-based approaches nor the model-based System MR Intent: request Slot-value pairs: [city =?] Generated Text which city are you interested in? NLG Figure 1: An example of NLG task. The model takes in the system MR, which consists of an intent with slot value pairs, and outputs text in natural language. approaches are sufficiently scalable for large-scale conversational systems, where it is common to have over hundreds of intents and thousands of slots. With the rise of neural transfer learning for NLP using pretrained LMs, recently, neural NLGs started to leverage transfer learning and showed some promising results (Radford et al., 2019; Brown et al., 2020; Dai et al., 2019; Edunov et al., 2019). In particular, Peng et al. (2020) proposed FEWSHOTWOZ, the first NLG benchmark test in few-shot learning settings, and achieved a SOTA performance by leveraging existing MR-to-Text data sets via task-specific continued pre-training. Despite the improved result, their approach leaves little room for further improvements as MR-to-Text data are expensive to obtain for new domains, prac- tically …

Content truncated to 500 words