Extracting Latent Steering Vectors from Pretrained Language Models

Findings of the Association for Computational Linguistics: ACL 2022, pages 566 - 581 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Extracting Latent Steering Vectors from Pretrained Language Models Nishant Subramani† Nivedita Suresh♢ Matthew E. Peters† †Allen Institute for Artificial Intelligence, Seattle, WA, USA ♢Arrive Bio, San Francisco, CA, USA {nishants,matthewp}@allenai.org {nive}@arrivebio.com Abstract Prior work on controllable text generation has focused on learning how to control language models through trainable decoding, smart- prompt design, or fine-tuning based on a de- sired objective. We hypothesize that the infor- mation needed to steer the model to generate a target sentence is already encoded within the model. Accordingly, we explore a different approach altogether: extracting latent vectors directly from pretrained language model de- coders without fine-tuning. Experiments show that there exist steering vectors, which, when added to the hidden states of the language model, generate a target sentence nearly per- fectly (> 99 BLEU) for English sentences from a variety of domains. We show that vector arith- metic can be used for unsupervised sentiment transfer on the Yelp sentiment benchmark, with performance comparable to models tailored to this task. We find that distances between steer- ing vectors reflect sentence similarity when evaluated on a textual similarity benchmark (STS-B), outperforming pooled hidden states of models. Finally, we present an analysis of the intrinsic properties of the steering vectors. Taken together, our results suggest that frozen LMs can be effectively controlled through their latent steering space. 1 1 Introduction Leveraging large pretrained language models trained on massive Web corpora has become the go-to approach to solve natural language process- ing tasks (Peters et al., 2018; Radford et al., 2018; Devlin et al., 2018; Brown et al., 2020). As a result, controlling these models has become paramount as many applications of NLP technology require con- trol over the generations of the model. Prior work aims to learn how to control language models and falls in three categories: trainable decoding (Gu 1Code is available at https://github.com/nis hantsubramani/steering_vectors. et al., 2017; Deng et al., 2020), smart-prompt de- sign (Shin et al., 2020; Lester et al., 2021), and fine-tuning based on a desired objective (Krause et al., 2021; Weng, 2021). Further, many works opt to train auto-encoder based models for controllable text generation (Shen et al., 2017, 2020; Mai et al., 2020). These approaches make controllability eas- ier by learning a latent space that is more easily manipulated to encourage models to generate text corresponding to a target attribute such as positive sentiment in the case of sentiment transfer. We take a more direct approach and explore whether it is possible to extract latent vectors di- rectly from pretrained language model decoders without fine-tuning. We call these vectors steering vectors and define the latent steering space of a sentence under a language model by the set of ex- tracted steering vectors, which steer the model to generate that sentence exactly. During decoding, we add our steering vector to the hidden states of the language model to generate the target …

Content truncated to 500 words