Sequence Level Knowledge Distillation
Related Papers
Sequence-Level Knowledge Distillation
Neural machine translation (NMT) offers a novel alternative formulation of translation that is potentially simpler than statistical ap- proaches. However to reach competitive per- formance, NMT models need to be …...
NAIST English-to-Japanese Simultaneous Translation System for IWSLT 2021 Simultaneous Text-to-text Task
AbstractThis paper describes NAIST’s system for the English-to-Japanese Simultaneous Text-to-text Translation Task in IWSLT 2021 Evaluation Campaign. Our primary submission is based on wait-k neural machine translation with sequence-level knowledge …...
POS-Constrained Parallel Decoding for Non-autoregressive Generation
AbstractThe multimodality problem has become a major challenge of existing non-autoregressive generation (NAG) systems. A common solution often resorts to sequence-level knowledge distillation by rebuilding the training dataset through autoregressive …...
One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation
AbstractNon-autoregressive neural machine translation (NAT) suffers from the multi-modality problem: the source sentence may have multiple correct translations, but the loss function is calculated only according to the reference sentence. …...
The RoyalFlush System for the WMT 2022 Efficiency Task
AbstractThis paper describes the submission of the RoyalFlush neural machine translation system for the WMT 2022 translation efficiency task. Unlike the commonly used autoregressive translation system, we adopted a two-stage …...