Twist Decoding: Diverse Generators Guide Each Other

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4909 - 4923 December 7-11, 2022 ©2022 Association for Computational Linguistics Twist Decoding: Diverse Generators Guide Each Other Jungo Kasai♡♣ Keisuke Sakaguchi♢∗ Ronan Le Bras♣ Hao Peng♣∗ Ximing Lu♡♣ Dragomir Radev♠ Yejin Choi♡♣ Noah A. Smith♡♣ ♡Paul G. Allen School of Computer Science & Engineering, University of Washington ♣Allen Institute for AI ♢Tohoku University ♠Department of Computer Science, Yale University {jkasai,lux32,yejin,nasmith}@cs.washington.edu keisuke.sakaguchi@tohoku.ac.jp {ronanlb,haop}@allenai.org dragomir.radev@yale.edu Abstract Many language generation models are now available for a wide range of generation tasks, including machine translation and summariza- tion. Combining such diverse models may lead to further progress, but ensembling gener- ation models is challenging during inference: conventional ensembling methods (e.g., shal- low fusion) require that the models share vo- cabulary/tokenization schemes. We introduce TWIST decoding, a simple and general text generation algorithm that benefits from diverse models at inference time. Our method does not assume the vocabulary, tokenization or even generation order is shared. Our exten- sive evaluations on machine translation and scientific paper summarization demonstrate that TWIST decoding substantially outperforms each model decoded in isolation over vari- ous scenarios, including cases where domain- specific and general-purpose models are both available. TWIST decoding also consistently outperforms the popular reranking heuristic where output candidates from one model are rescored by another. We hope that our work will encourage researchers and practitioners to examine generation models collectively, not just independently, and to seek out models with complementary strengths to the currently avail- able models.1 1 Introduction Natural language generation is an important build- ing block for many applications, such as machine translation, summarization, and question answer- ing (Ng et al., 2019; Lewis et al., 2020; Raffel et al., 2020; Brown et al., 2020; Asai et al., 2021, inter alia). Researchers have recently explored and ad- vanced models for generation in various aspects, in- ∗ This work was done while Keisuke Sakaguchi was at the Allen Institute for AI and Hao Peng was at the University of Washington. 1Our code is available at https://github.com/ jungokasai/twist_decoding. BOS BOS BOS COVID cases in@ clude ... COV@ ID cases include ... COV@ ID symptoms include ... COVID symp@ toms in@ ... COVID symp@ toms in@ ... Figure 1: TWIST decoding of two generation models, f and g, that does not assume a shared vocabulary, tok- enization, or generation order. Beam search is first ap- plied to f to generate y(0), followed by output mapping to ˜︁y(0) (e.g., f’s detokenization and g’s tokenization or sequence reversal). g is then decoded with beam search augmented with distances from the set of previously- generated outputs (here only one sequence y is shown): d(z(1) ≤n, ˜︁y(0) ≤n). Subsequently, f is similarly decoded with g’s guidance. Here we show one iteration that already achieves substantial improvements (§4). @ indicates the BPE separator. cluding model architecture (Bahdanau et al., 2015; Vaswani et al., 2017), domain adaptation (Chu and Wang, 2018; Bapna and Firat, 2019), prompting (Brown et al., 2020), and even generation …

Content truncated to 500 words