FUDGE: Controlled Text Generation With Future Discriminators

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3511–3535 June 6–11, 2021. ©2021 Association for Computational Linguistics 3511 FUDGE: Controlled Text Generation With Future Discriminators Kevin Yang UC Berkeley yangk@berkeley.edu Dan Klein UC Berkeley klein@berkeley.edu Abstract We propose Future Discriminators for Genera- tion (FUDGE), a flexible and modular method for controlled text generation. Given a pre- existing model G for generating text from a dis- tribution of interest, FUDGE enables condition- ing on a desired attribute a (for example, for- mality) while requiring access only to G’s out- put logits. FUDGE learns an attribute predictor operating on a partial sequence, and uses this predictor’s outputs to adjust G’s original prob- abilities. We show that FUDGE models terms corresponding to a Bayesian decomposition of the conditional distribution of G given attribute a. Moreover, FUDGE can easily compose pre- dictors for multiple desired attributes. We eval- uate FUDGE on three tasks — couplet comple- tion in poetry, topic control in language gener- ation, and formality change in machine trans- lation — and observe gains in all three tasks. 1 Introduction Recent advances in large pretrained language mod- els allow us to generate increasingly realistic text by modeling a distribution P(X) over natural lan- guage sequences X. The distribution P(X) may be truly unconditional, as is common in language modeling, or it may model P(X|I) conditioned on some input I, as in machine translation or summa- rization. We are frequently interested in controlled text generation, the task of generating text conditioned on an additional desirable attribute a which is not already built into P(X). That is, we would like to model P(X|a) (or possibly P(X|I, a); henceforth we will drop I from the notation for simplicity). For example, P(X) may be a pretrained transla- tion model for Spanish inputs I to English outputs X, but we may wish to additionally constrain the outputs to possess a new attribute a, e.g., formality, which we did not optimize for during training. Unfortunately, once we have already obtained an unconditioned P(X) defined as the output dis- tribution of some large generative model G, it is nontrivial to add conditioning on a new attribute a without either training a new model from scratch or fine-tuning with additional data. Although in principle we can trivially sample from P(X|a) via rejection sampling from P(X), rejection sampling may be highly inefficient in practice. On the other hand, while generating according to attribute a, P(X) should be left otherwise intact: in the pre- vious translation formality example, it is pointless to generate formal English outputs if they do not preserve the original Spanish meaning. In light of these concerns, we propose Future Discriminators for Generation (FUDGE), a flexible and modular method for modeling P(X|a) which accesses only the output probabilities of the gener- ative model G which defines P(X). FUDGE learns a binary predictor for whether attribute a will be- come true in the complete future, based on an …

Content truncated to 500 words