Coherence boosting: When your pretrained language model is not paying enough attention

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8214 - 8236 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Coherence boosting: When your pretrained language model is not paying enough attention Nikolay Malkin Mila / Université de Montréal nikolay.malkin@mila.quebec Zhen Wang Ohio State University wang.9215@osu.edu Nebojsa Jojic Microsoft Research jojic@microsoft.com Abstract Long-range semantic coherence remains a challenge in automatic language generation and understanding. We demonstrate that large language models have insufficiently learned the effect of distant words on next-token pre- diction. We present coherence boosting, an in- ference procedure that increases a LM’s focus on a long context. We show the benefits of coherence boosting with pretrained models by distributional analyses of generated ordinary text and dialog responses. It is also found that coherence boosting with state-of-the-art mod- els for various zero-shot NLP tasks yields per- formance gains with no additional training. 1 Introduction Language models (LMs) are commonly evaluated for their ability to generate, rank, or classify co- herent spans of text. Long-range semantic coher- ence is a unifying feature of modern NLP bench- marks and applications, whether they are about pro- ducing short answers to questions, ranking answer choices by their consistency with world knowledge, or generating long responses. Large nonspecialized LMs, such as GPT-2 and -3 (Radford et al., 2019; Brown et al., 2020), some- times fail to understand or use the semantic link between a text and its prompt or long-range context (Fig. 1). Samples from these LMs have an unnatu- rally low density of words that require many tokens of context to predict (§4.1), and the scores that the models give to completions of prompts indicate that they are oversensitive to recent context (§5). We hypothesize that these failures arise from modeling choices and distribution shift. Specif- ically, autoregressive LMs are typically fit to a multi-objective problem: simultaneously maximiz- ing token likelihoods conditioned on many lengths of truncated context (§2.1). Yet, at generation or Code: github.com/zhenwang9102/coherence-boosting. scoring time, likelihoods are conditioned on the en- tire prompt or previously generated string, specif- ically selected to be coherent or even guaranteed to influence the output. The two common solu- tions – finetuning models on one or multiple tasks (Khashabi et al., 2020; Sanh et al., 2022) and im- proving models or prompts to facilitate in-context learning (Brown et al., 2020; Schick and Schütze, 2021) – do not directly target the problem of long- range coherence. This paper proposes coherence boosting, a sim- ple inference-time procedure that increases the ef- fect of distant words on predicted token distribu- tions and is applicable in both generation and rank- ing settings. A pretrained model is viewed as an en- semble of experts that produce token distributions conditioned on varying lengths of context. These experts are log-linearly mixed to form a predictor that is superior to the base model (§2). Coherence boosting greatly improves prediction of words that depend on a long context, as evi- denced by state-of-the-art results on tasks specially …

Content truncated to 500 words