EEL: Efficiently Encoding Lattices for Reranking

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9299–9316 July 9-14, 2023 ©2023 Association for Computational Linguistics EEL: Efficiently Encoding Lattices for Reranking Prasann Singhal♢ Jiacheng Xu♠ Xi Ye♢ Greg Durrett♢ ♢The University of Texas at Austin, ♠Salesforce AI {prasanns, xiye, gdurrett}@cs.utexas.edu, jiacheng.xu@salesforce.com Abstract Standard decoding approaches for conditional text generation tasks typically search for an out- put hypothesis with high model probability, but this may not yield the best hypothesis accord- ing to human judgments of quality. Reranking to optimize for downstream metrics can better optimize for quality, but many metrics of in- terest are computed with pre-trained language models, which are slow to apply to large num- bers of hypotheses. We explore an approach for reranking hypotheses by using Transformers to efficiently encode lattices of generated outputs, a method we call EEL. With a single Trans- former pass over the entire lattice, we can ap- proximately compute a contextualized represen- tation of each token as if it were only part of a single hypothesis in isolation. We combine this approach with a new class of token-factored rerankers (TFRs) that allow for efficient extrac- tion of high reranker-scoring hypotheses from the lattice. Empirically, our approach incurs minimal degradation error compared to the ex- ponentially slower approach of encoding each hypothesis individually. When applying EEL with TFRs across three text generation tasks, our results show both substantial speedup com- pared to naive reranking and often better perfor- mance on downstream metrics than comparable approaches.1 1 Introduction Part of the progress in natural language genera- tion over the past few years has been driven by a proliferation of decoding techniques, from beam search to sampling approaches like nucleus sam- pling (Holtzman et al., 2020), typical decoding (Meister et al., 2022), and contrastive decoding (Li et al., 2022). These techniques, however, only optimize for probabilistic objectives, rather than alignment with human judgments, which is typi- cally better encapsulated by downstream metrics 1Code available at https://github.com/PrasannS/ eel-reranking. Il fait beau Reranking (Prior work) The weather is nice Transformer reranker The weather is beautiful Transformer reranker It is nice Transformer reranker Explicitly score candidates, choose highest score s4 s3 s2 The weather It is nice beautiful EEL Transformer (causal w/mask) Compute reranker score for each node Token-factored reranking of lattices (This work) The weather is nice beautiful It It is beautiful Generation model Efficiently extract highest scoring path Transformer reranker s1 0.9 0.3 0.8 0.9 0.2 0.8 The weather is beautiful Lattice of generation outputs Figure 1: Overview of our approach. For a task such as translation, we can generate a lattice of plausible model outputs. Reranking these outputs with Transformers can be slow; we can achieve speedups by efficiently encoding the lattice in a single Transformer pass and using a token-factored reranker to efficiently find the best hypothesis in the lattice. (Zhang et al., 2019b; Dhingra et al., 2019; Sellam et al., 2020; Rei et al., 2020) that specifically es- timate human preference. Transformer …

Content truncated to 500 words