R-TeaFor: Regularized Teacher-Forcing for Abstractive Summarization

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6303 - 6311 December 7-11, 2022 ©2022 Association for Computational Linguistics R-TeaFor: Regularized Teacher-Forcing for Abstractive Summarization Guan-Yu Lin National Taiwan University r09944017@ntu.edu.tw Pu-Jen Cheng National Taiwan University pjcheng@csie.ntu.edu.tw Abstract Teacher-forcing is widely used in training se- quence generation models to improve sampling efficiency and to stabilize training. However, teacher-forcing is vulnerable to the exposure bias problem. Previous works have attempted to address exposure bias by modifying the train- ing data to simulate model-generated results. Nevertheless, they do not consider the pairwise relationship between the original training data and the modified ones, which provides more information during training. Hence, we pro- pose Regularized Teacher-Forcing (R-TeaFor) to utilize this relationship for better regulariza- tion. Empirically, our experiments show that R-TeaFor outperforms previous summarization state-of-the-art models, and the results can be generalized to different pre-trained models. 1 Introduction Recently, the encoder-decoder models have made a giant leap in sequence generation tasks. These mod- els are primarily trained with the teacher-forcing method (Goodfellow et al., 2016), which has been praised for enhancing sampling efficiency and train- ing stability. However, as teacher-forcing only exposes models to the training data distribution rather than their prediction distribution, models suf- fer from the exposure bias problem (Bengio et al., 2015; Ranzato et al., 2015). Previous works for solving exposure bias can be roughly split into four categories: regularization, contrastive learning, reinforcement learning, and sampling-based methods. Regularization methods such as dropout (Srivastava et al., 2014) and pertur- bation (Goodfellow et al., 2015) focus on problems of data distribution differences between training data and inference data, which is also related to exposure bias. Liang et al. (2021) propose R-Drop, which refines dropout by forcing predictions of dif- ferent sub-models generated by dropout to be con- sistent with each other. Gal and Ghahramani (2016) Figure 1: An illustration of R-TeaFor. The output dis- tributions are generated both by the original target se- quence and the augmented target sequence. R-TeaFor aims to reduce the distance of two distributions by addi- tional symmetric KL-divergence loss. apply token-level word dropout in language mod- eling. Takase and Kiyono (2021) then introduce word dropout to sequence generation. Other token- level perturbations include replacing tokens with source tokens, model-generated tokens (Bengio et al., 2015), and a mixture of them (Zhang et al., 2019). Adversarial perturbation (Sato et al., 2019; Aghajanyan et al., 2021) is another alternative that directly modifies the token embedding. Although widely used, most regularization methods are not exclusively designed for the exposure bias problem. Hence, their ability to alleviate exposure bias is limited. Contrastive learning is widely used in either pre- training or fine-tuning sequence generation models to provide more information inside the training data. Word vectors (Mikolov et al., 2013) and lan- guage models are the early applications for con- trastive learning in natural language processing. 6303 For summarization tasks, both extractive-based ap- proach (Zhong et al., 2020) and abstractive-based approach can be combined with contrastive learn- ing. These …

Content truncated to 500 words