Экспозициялық ауытқу
Related Papers
An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation
AbstractGenerating paraphrases from given sentences involves decoding words step by step from a large vocabulary. To learn a decoder, supervised learning which maximizes the likelihood of tokens always suffers from …...
Autoregressive Knowledge Distillation through Imitation Learning
AbstractThe performance of autoregressive models on natural language generation tasks has dramatically improved due to the adoption of deep, self-attentive architectures. However, these gains have come at the cost of …...
Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation
AbstractKnowledge selection plays an important role in knowledge-grounded dialogue, which is a challenging task to generate more informative responses by leveraging external knowledge. Recently, latent variable models have been proposed …...
Minimize Exposure Bias of Seq2Seq Models in Joint Entity and Relation Extraction
AbstractJoint entity and relation extraction aims to extract relation triplets from plain text directly. Prior work leverages Sequence-to-Sequence (Seq2Seq) models for triplet sequence generation. However, Seq2Seq enforces an unnecessary order …...
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation
AbstractThe standard training algorithm in neural machine translation (NMT) suffers from exposure bias, and alternative algorithms have been proposed to mitigate this. However, the practical impact of exposure bias is …...
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation?
AbstractExposure bias has been regarded as a central problem for auto-regressive language models (LM). It claims that teacher forcing would cause the test-time generation to be incrementally distorted due to …...
TILGAN: Transformer-based Implicit Latent GAN for Diverse and Coherent Text Generation
Conventional autoregressive models have achieved great success in text generation but suffer from the exposure bias problem in that token sequences in the training and in the gen- eration stages …...
Type-Driven Multi-Turn Corrections for Grammatical Error Correction
AbstractGrammatical Error Correction (GEC) aims to automatically detect and correct grammatical errors. In this aspect, dominant models are trained by one-iteration learning while performing multiple iterations of corrections during inference. …...
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
AbstractCurrent language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis for this brittleness of generation models is that it is caused by the training …...
Understanding and Bridging the Modality Gap for Speech Translation
AbstractHow to achieve better end-to-end speech translation (ST) by leveraging (text) machine translation (MT) data? Among various existing techniques, multi-task learning is one of the effective ways to share knowledge …...