Selective Token Generation for Few-shot Natural Language Generation

Proceedings of the 29th International Conference on Computational Linguistics, pages 5837–5856 October 12–17, 2022. 5837 Selective Token Generation for Few-shot Natural Language Generation Daejin Jo1, Taehwan Kwon1, Eun-Sol Kim2, and Sungwoong Kim1 1Kakao Brain 2Hanyang University 1{daejin.jo, taehwan.kwon, swkim}@kakaobrain.com 2eunsolkim@hanyang.ac.kr Abstract Natural language modeling with limited train- ing data is a challenging problem, and many algorithms make use of large-scale pretrained language models (PLMs) for this due to its great generalization ability. Among them, ad- ditive learning that incorporates a task-specific adapter on top of the fixed large-scale PLM has been popularly used in the few-shot setting. However, this added adapter is still easy to dis- regard the knowledge of the PLM especially for few-shot natural language generation (NLG) since an entire sequence is usually generated by only the newly trained adapter. Therefore, in this work, we develop a novel additive learn- ing algorithm based on reinforcement learning (RL) that selectively outputs language tokens between the task-general PLM and the task- specific adapter during both training and infer- ence. This output token selection over the two generators allows the adapter to take into ac- count solely the task-relevant parts in sequence generation, and therefore makes it more robust to overfitting as well as more stable in RL train- ing. In addition, to obtain the complementary adapter from the PLM for each few-shot task, we exploit a separate selecting module that is also simultaneously trained using RL. Experi- mental results on various few-shot NLG tasks including question answering, data-to-text gen- eration and text summarization demonstrate that the proposed selective token generation significantly outperforms the previous additive learning algorithms based on the PLMs. 1 Introduction Recently, pretrained language models (PLMs) have shown great generalization ability when combined with large-scale data and big transformer-based models (Devlin et al., 2019; Radford et al., 2019; Lewis et al., 2020; Brown et al., 2020; Subra- manyam Kalyan et al., 2021; Petroni et al., 2019; Wang et al., 2020). Therefore, transfer learn- ing from PLMs has been popularly used for few- shot natural language generation (NLG) tasks with promising results. In specific, the use of PLM for few-shot NLG can be categorized into three ap- proaches: 1) prompt-based, 2) finetuning, and 3) ad- ditive learning. Prompt-based approaches encode a task description and task-specific examples as a nat- ural language prompt for few-shot text generation (Radford et al., 2019; Brown et al., 2020; Zheng and Huang, 2021; Schick and Schütze, 2020; Li and Liang, 2021a). These approaches can take full advantage of the universal natural language under- standing and generation capabilities of large-scale PLMs without further training of the main model, however, they have some limitations in dealing with a large domain shift from the pretraining corpus data, tuning suitable task-specific prompts, and cov- ering an increased size of conditioning examples. On the other hand, finetuning of the PLM is able to explicitly impart task-specific knowledge to the model and hence lift the above limitations (Ziegler et al., 2019; Xu et al., 2021; Chen et al., 2020). However, …

Content truncated to 500 words