Towards Climate Awareness in NLP Research

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 199 - 232 December 7-11, 2022 ©2022 Association for Computational Linguistics RANKGEN: Improving Text Generation with Large Ranking Models Kalpesh Krishna♠ ∗ Yapei Chang♠ John Wieting♦ Mohit Iyyer♠ ♠University of Massachusetts Amherst, ♦Google Research {kalpesh,miyyer}@cs.umass.edu jwieting@google.com Abstract Given an input sequence (or prefix), mod- ern language models often assign high prob- abilities to output sequences that are repeti- tive, incoherent, or irrelevant to the prefix; as such, model-generated text also contains such artifacts. To address these issues we present RANKGEN, a 1.2B parameter encoder model for English that scores model genera- tions given a prefix. RANKGEN can be flexi- bly incorporated as a scoring function in beam search and used to decode from any pretrained language model. We train RANKGEN using large-scale contrastive learning to map a prefix close to the ground-truth sequence that follows it and far away from two types of negatives: (1) random sequences from the same docu- ment as the prefix, and (2) sequences gener- ated from a large language model conditioned on the prefix. Experiments across four differ- ent language models (345M-11B parameters) and two domains show that RANKGEN signif- icantly outperforms decoding algorithms like nucleus, top-k, and typical sampling on both automatic metrics (85.0 vs 77.3 MAUVE) as well as human evaluations with English writ- ers (74.5% human preference over nucleus sampling). Analysis reveals that RANKGEN outputs are more relevant to the prefix and im- prove continuity and coherence compared to baselines. We release our model checkpoints, code, and human preference data with explana- tions to facilitate future research.1 1 Introduction Despite exciting recent progress in large-scale lan- guage modeling (Radford et al., 2019; Brown et al., 2020), text generated from these language mod- els (LMs) continues to be riddled with artifacts. Modern LMs suffer from the “likelihood trap” (See et al., 2019; Zhang et al., 2021), in which high 1All resources are available at https://github. com/martiansideofthemoon/rankgen. *Work done as a student researcher at Google Research. likelihood (low perplexity) sequences produced by greedy decoding or beam search tend to be dull and repetitive. While truncated sampling methods such as top-k (Fan et al., 2018), nucleus (Holtzman et al., 2020), and typical sampling (Meister et al., 2022) alleviate these issues, they can also produce text with inconsistencies, hallucinations, factual errors, or commonsense issues (Massarelli et al., 2020; Dou et al., 2022; Krishna et al., 2021). Part of the problem is that LMs are trained using “teacher forcing”, where they are always given the ground-truth prefix2 and asked to predict the next token. At test-time, however, the prefix can contain model-generated text, allowing errors to propagate during decoding (Bengio et al., 2015). This issue, combined with the observation that LMs overly rely on local context (Khandelwal et al., 2018; Sun et al., 2021), contributes to the generation of sequences that break coherence or consistency within a larger discourse-level context (Wang et al., 2022). To address this issue we present RANKGEN, a 1.2 billion …

Content truncated to 500 words