Massive-scale Decoding for Text Generation using Lattices

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4659 - 4676 July 10-15, 2022 ©2022 Association for Computational Linguistics Massive-scale Decoding for Text Generation using Lattices Jiacheng XuN Siddhartha Reddy Jonnalagadda# Greg DurrettN NDepartment of Computer Science, The University of Texas at Austin #Alexa AI, Amazon, Seattle {jcxu,gdurrett}@cs.utexas.edu sidjreddy@gmail.com Abstract Conditional neural text generation models gen- erate high-quality outputs, but often concen- trate around a mode when what we really want is a diverse set of options. We present a search algorithm to construct lattices encod- ing a massive number of generation options. First, we restructure decoding as a best-first search, which explores the space differently than beam search and improves efficiency by avoiding pruning paths. Second, we revisit the idea of hypothesis recombination: we can identify pairs of similar generation candidates during search and merge them as an approxi- mation. On both summarization and machine translation, we show that our algorithm en- codes thousands of diverse options that remain grammatical and high-quality into one lattice. This algorithm provides a foundation for build- ing downstream generation applications on top of massive-scale diverse outputs.1 1 Introduction Although pre-trained text generation models (Lewis et al., 2020; Raffel et al., 2020) have achieved impressive results across a range of tasks, these models do not always deliver what system developers want. Machine generated text may be non-factual (Kryscinski et al., 2020; Maynez et al., 2020; Goyal and Durrett, 2021) or toxic (Gehman et al., 2020). We might patch these problems by applying discriminators over the output (Holtzman et al., 2018; Yang and Klein, 2021) to enforce these properties post-hoc; we could, for instance, apply a secondary model as a reranker over a small collec- tion of outputs. However, if the generator returns a homogeneous set of candidates, we may fail to find any usable generation output. What if generation models could return massive numbers of candi- dates rather than a few outputs with optimal score? 1Code, implementation guideline, and visualization are available at https://github.com/jiacheng-xu/ lattice-generation. single hypothesis to proceed with lattice encodes 4 variations compactly Beam Search (high redundancy in outputs, hypotheses duplicated) A Cardiff recycling company has gone into bankruptcy. A Cardiff waste management company has gone into bankruptcy. A Cardiff waste processing company has gone into bankruptcy. A Cardiff waste processing plant has gone into bankruptcy. A recycling company has waste management processing plant Cardiff gone into bankruptcy. This Work: Search with Recombination Model Figure 1: A lattice of outputs yielded by path recombi- nation is a more efficient way to represent and explore related generation outputs compared to beam search. With a large set of candidates, our secondary model could more easily find an acceptable one without having to take more extreme steps like re-training the initial generation model. Output diversity has separately been established as a useful goal for for applications such as dialogue and story generation (Li et al., 2016; Fan et al., 2019). Standard approaches including …

Content truncated to 500 words