The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 10406–10420 July 9-14, 2023 ©2023 Association for Computational Linguistics The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, Eyal Shnarch IBM Research {ariel.gera1, ofir.arviv, chulaka.gunasekara}@ibm.com, {roni.friedman-melamed, benjams, noams, eyals}@il.ibm.com Abstract Applying language models to natural language processing tasks typically relies on the repre- sentations in the final model layer, as inter- mediate hidden layer representations are pre- sumed to be less informative. In this work, we argue that due to the gradual improvement across model layers, additional information can be gleaned from the contrast between higher and lower layers during inference. Specifically, in choosing between the probable next token predictions of a generative model, the predic- tions of lower layers can be used to highlight which candidates are best avoided. We propose a novel approach that utilizes the contrast be- tween layers to improve text generation outputs, and show that it mitigates degenerative behav- iors of the model in open-ended generation, sig- nificantly improving the quality of generated texts. Furthermore, our results indicate that contrasting between model layers at inference time can yield substantial benefits to certain aspects of general language model capabilities, more effectively extracting knowledge during inference from a given set of model parameters. 1 Introduction For a wide range of natural language processing tasks, the standard practice is to rely on deep neu- ral networks with a transformer-based architecture (Vaswani et al., 2017). Such models are composed of multiple transformer layers, where typically the representations of the final layer are used for the downstream task. As shown in prior works, some of the representational knowledge required for per- forming downstream tasks can already be found within intermediate layers of the model (Geva et al., 2021, 2022); at the same time, relying on the rep- resentations of lower model layers does result in decreased performance, specifically for inputs that are more challenging (Schwartz et al., 2020; Xin et al., 2020; Elbayad et al., 2020; Sun et al., 2022; Schuster et al., 2022; Din et al., 2023). Figure 1: An example of auto-contrastive decoding (ACD) with GPT2, where the top layer (24) is taken as the expert and contrasted with layer 12, the amateur. As decoding is done token by token, we can only see the direct effect on the first token, where ACD leads to selecting an alternative high probability token - "In". Recently, Li et al. (2022) considered a scenario involving two language models; one is a very large pre-trained model, termed the expert, and the other is a much smaller version of the same architecture, termed the amateur. Importantly, whereas these models share some failure modes and undesirable behaviors, the expert model clearly outperforms the amateur model in language model tasks. Focus- ing on an open-ended auto-regressive text gener- ation task, they show that it is possible to exploit the contrast between the predictions of the …

Content truncated to 500 words