Improving Controllable Text Generation with Position-Aware Weighted Decoding

Findings of the Association for Computational Linguistics: ACL 2022, pages 3449 - 3467 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Improving Controllable Text Generation with Position-Aware Weighted Decoding Yuxuan Gu†, Xiaocheng Feng†‡, Sicheng Ma†, Jiaming Wu†, Heng Gong†, Bing Qin†‡ †Harbin Institute of Technology ‡ Peng Cheng Laboratory {yxgu,xcfeng,scma,jmwu,hgong,bqin}@ir.hit.edu.cn Abstract Weighted decoding methods composed of the pretrained language model (LM) and the con- troller have achieved promising results for con- trollable text generation. However, these mod- els often suffer from a control strength/fluency trade-off problem as higher control strength is more likely to generate incoherent and repeti- tive text. In this paper, we illustrate this trade- off is arisen by the controller imposing the tar- get attribute on the LM at improper positions. And we propose a novel framework based on existing weighted decoding methods called CAT-PAW1, which introduces a lightweight reg- ulator to adjust bias signals from the controller at different decoding positions. Experiments on positive sentiment control, topic control, and language detoxification show the effectiveness of our CAT-PAW upon 4 SOTA models2. 1 Introduction Controllable text generation is a challenging task in natural language generation, which aims to gener- ate diverse text related to specified attributes. Domi- nating studies follow PPLM (Dathathri et al., 2020) and adopt a weighted decoding strategy (Krause et al., 2020; Yang and Klein, 2021; Liu et al., 2021a). They usually employ an external controller with weight λ to bias the output distribution of a fixed pretrain LM. And the weight λ is positively correlated to control strength, thereby achieving strength-adjustable controllable text generation. However, those weighted decoding methods suffer from a trade-off problem between control strength and text fluency. As illustrated in Figure 1, when control strength increases, fluency of text generated by these SOTA models such as PPLM (Dathathri et al., 2020), Fudge (Yang and Klein, 2021), GeDi (Krause et al., 2020), and DExperts 1CAT-PAW stands for ControllAble Text generation with Position-Aware Weighted decoding. 2Our dataset and code are available at: https://gi thub.com/hit-scma/CAT-PAW. 40 50 60 70 80 PPLM DExperts 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 40 50 60 70 80 Fudge 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 GeDi Control Strength → Perplexit ← Figure 1: Trade-off between control strength and text fluency on positive sentiment control, where control strength is the probability of being positive and perplex- ity is an inversely proportional metric to fluency. Each point represents results sampled from an individual λ. (Liu et al., 2021a) will drop rapidly. In addition, cases in Figure 2 shows that with the increase of weight λ from 0.03 to 0.09, models are more likely to degenerate with repetitive, contradictory and in- coherent contents such as “it was war war for war”. Therefore, it’s vital to alleviate the trade-off as an ideal controllable generator should generate high- quality text under different control strengths. Based on our analysis, the trade-off is due to the controller assigning bias signals to all decod- ing positions while ignoring the original results of …

Content truncated to 500 words