Improving Text Generation with Student-Forcing Optimal Transport

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 9144–9156, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 9144 Improving Text Generation with Student-Forcing Optimal Transport Jianqiao Li∗1, Chunyuan Li∗2, Guoyin Wang∗3, Hao Fu1, Yuh-Chen Lin3 Liqun Chen1, Yizhe Zhang2, Chenyang Tao1, Ruiyi Zhang1 Wenlin Wang1, Dinghan Shen4, Qian Yang1 and Lawrence Carin1 1Duke University 2Microsoft Research 3Amazon 4Microsoft Dynamics 365 AI jianqiao.li@duke.edu, chunyl@microsoft.com, guoyiwan@amazon.com Abstract Neural language models are often trained with maximum likelihood estimation (MLE), where the next word is generated conditioned on the ground-truth word tokens. During test- ing, however, the model is instead conditioned on previously generated tokens, resulting in what is termed exposure bias. To reduce this gap between training and testing, we propose using optimal transport (OT) to match the se- quences generated in these two modes. An ex- tension is further proposed to improve the OT learning, based on the structural and contex- tual information of the text sequences. The effectiveness of the proposed method is vali- dated on machine translation, text summariza- tion, and text generation tasks. 1 Introduction Natural language generation is an essential com- ponent of many NLP applications, such as ma- chine translation (Bahdanau et al., 2015), image captioning (You et al., 2016), text summarization (See et al., 2017), dialogue systems (Vinyals and Le, 2015), and machine comprehension (Nguyen et al., 2016). Generating human-like natural lan- guage is typically cast as predicting a sequence of consecutive words in a recurrent manner. Max- imum likelihood estimation (MLE) is commonly employed as the objective to train such text- generation models, maximizing the log-likelihood of producing the ground-truth tokens within a sen- tence or paragraph (Salakhutdinov, 2015). In Re- current Neural Network (RNN) models, this is also known as Teacher-Forcing (TF) (Williams and Zipser, 1989), due to the use of ground-truth tokens for next-token prediction. However, in the maximum likelihood paradigm, previous observed tokens are usually provided during training, giving rise to an issue termed ex- posure bias (Bengio et al., 2015): the ground-truth ∗* Equal contribution tokens seen by the model during training are not available at inference time. During inference, the model is required to use outputs from the last step instead of the unseen ground-truth, which is often referred to as Student-Forcing (SF). As a result, there is a discrepancy between training and in- ference, accumulating errors along the sequence- generation trajectory (Ranzato et al., 2016a). This challenge has been addressed by incorpo- rating model-generated text at training time. Ben- gio et al. (2015) proposed scheduled sampling (SS), where the training samples are systemat- ically tampered with by replacing some of the ground-truth tokens with model-predicted tokens. Other works regard text generation as a sequential decision making problem, applying reinforcement learning (RL) techniques (Ranzato et al., 2016b; Bahdanau et al., 2017). In particular, quantita- tive evaluation metrics such as BLEU and ROGUE are used as sequence-level rewards for model- generated texts. Despite the encouraging results reported, con- cerns have been raised w.r.t. the above strategies. …

Content truncated to 500 words