In-sample Curriculum Learning by Sequence Completion for Natural Language Generation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11937–11950 July 9-14, 2023 ©2023 Association for Computational Linguistics In-sample Curriculum Learning by Sequence Completion for Natural Language Generation Qi Jia1, Yizhu Liu2, Haifeng Tang3, Kenny Q. Zhu4∗ 1,4Shanghai Jiao Tong University, Shanghai, China 2Meituan, Shanghai, China 3China Merchants Bank Credit Card Center, Shanghai, China 1Jia_qi@sjtu.edu.cn, 2liuyizhu@meituan.com 3thfeng@cmbchina.com, 4kzhu@cs.sjtu.edu.cn Abstract Curriculum learning has shown promising im- provements in multiple domains by training machine learning models from easy samples to hard ones. Previous works which either design rules or train models for scoring the difficulty highly rely on task-specific expertise, and can- not generalize. Inspired by the “easy-to-hard” intuition, we propose to do in-sample curricu- lum learning for natural language generation tasks. Our learning strategy starts training the model to generate the last few words, i.e., do sequence completion, and gradually extends to generate the whole output sequence. Compre- hensive experiments show that it generalizes well to different tasks and achieves significant improvements over strong baselines. 1 Introduction Curriculum learning (CL) proposed by Bengio et al. (2009) provides performance improvements on a number of machine learning tasks. It mimics the learning process of humans by training models with samples in a more meaningful order, i.e., from the easy ones to the hard ones. Therefore, ranking training samples by difficulty lies in the core of CL, which is also the key challenge when it’s applied to natural language generation (NLG) tasks. Previous work on CL for NLG focuses on mea- suring the difficulty of training samples in two ways. One is to resort to human-crafted rules based on various linguistic features and human observations (Liu et al., 2018; Kocmi and Bo- jar, 2017). The other uses models either trained from outside data or the same data but in previous epochs/steps (Zhou et al., 2020; Kumar et al., 2019; Shen and Feng, 2020). Either way seeks to produce a numeric score for each training sample relying on domain expertise so that it can be ranked, mak- ing it difficult to generalize to different tasks. For ∗ The corresponding author. example, summarization focuses more on generat- ing concise outputs while style transfer emphasizes style changes. So the former should pay attention to the ratio between the lengths of the output and the input (the more compressed the more difficult), while the latter should focus on differences in style between the input and output (the more different the more difficult). Designing a comprehensive or universal scoring function is difficult or even impossible under this definition of CL. In this paper, we propose an alternative to sample-wise CL, which we call in-sample CL (ICL). ICL re-orders the learning sequence within the sample. One particular ICL re-ordering strat- egy which we find effective is to predict the last few tokens given a long prefix first from the origi- nal output, and then gradually increase the number of tokens at the end while shortening the prefix, to create an easy-to-hard training …

Content truncated to 500 words