Backpack Language Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9103–9125 July 9-14, 2023 ©2023 Association for Computational Linguistics Backpack Language Models John Hewitt John Thickstun Christopher D. Manning Department of Computer Science, Stanford UniversityPercy Liang {johnhew,jthickstun,manning,pliang}@cs.stanford.edu Abstract We present Backpacks: a new neural architec- ture that marries strong modeling performance with an interface for interpretability and con- trol. Backpacks learn multiple non-contextual sense vectors for each word in a vocabulary, and represent a word in a sequence as a context- dependent, non-negative linear combination of sense vectors in this sequence. We find that, after training, sense vectors specialize, each encoding a different aspect of a word. We can interpret a sense vector by inspecting its (non-contextual, linear) projection onto the out- put space, and intervene on these interpretable hooks to change the model’s behavior in pre- dictable ways. We train a 170M-parameter Backpack language model on OpenWebText, matching the loss of a GPT-2 small (124M- parameter) Transformer. On lexical similar- ity evaluations, we find that Backpack sense vectors outperform even a 6B-parameter Trans- former LM’s word embeddings. Finally, we present simple algorithms that intervene on sense vectors to perform controllable text gen- eration and debiasing. For example, we can edit the sense vocabulary to tend more towards a topic, or localize a source of gender bias to a sense vector and globally suppress that sense. 1 Introduction Consider the prefix The CEO believes that ___, and the problem of debiasing a neural language model’s distribution over he/she. Intuitively, the bias for he originates in the word CEO, because replacing CEO with nurse flips the observed bias. A success- ful intervention to debias CEO must reliably apply in all contexts in which the word CEO appears; ideally we would want to make a non-contextual change to the model that has predictable effects in all contexts. In general, in all aspects of in- terpretability and control, it is desirable to make interventions with a tractable interface (e.g., non- contextual representations) that apply globally. ∑ The tea is hot =hot * The tea is Transformer Transformer LM Transformer Backpack LM Figure 1: Transformers are monolithic functions of se- quences. In Backpacks, the output is a weighted sum of non-contextual, learned word aspects. Such interventions are difficult in Transformer models (Vaswani et al., 2017) because their con- textual representations are monolithic functions of their input. Almost any intervention on the model has complex, non-linear effects that depend on con- text. We would instead like models that enable precise, rich interventions that apply predictably in all contexts, and are still expressive, so they are a viable alternative to Transformers. We address these challenges with a new neu- ral architecture, the Backpack, for which predic- tions are log-linear combinations of non-contextual representations. We represent each word in a vo- cabulary as a set of non-contextual sense vectors that represent distinct learned aspects of the word. For example, sense vectors for the word “science” could encode types of science, …

Content truncated to 500 words