CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1023 - 1043 December 7-11, 2022 ©2022 Association for Computational Linguistics A Distributional Lens for Multi-Aspect Controllable Text Generation Yuxuan Gu†,Xiaocheng Feng†‡,Sicheng Ma†,Lingyuan Zhang†,Heng Gong†, Bing Qin†‡ †Harbin Institute of Technology ‡ Peng Cheng Laboratory {yxgu,xcfeng,scma,lyzhang,hgong,bqin}@ir.hit.edu.cn Abstract Multi-aspect controllable text generation is a more challenging and practical task than single- aspect control. Existing methods achieve com- plex multi-aspect control by fusing multiple controllers learned from single-aspect, but suf- fer from attribute degeneration caused by the mutual interference of these controllers. To address this, we provide observations on at- tribute fusion from a distributional perspective and propose to directly search for the intersec- tion areas of multiple attribute distributions as their combination for generation. Our method first estimates the attribute space with an au- toencoder structure. Afterward, we iteratively approach the intersections by jointly minimiz- ing distances to points representing different attributes. Finally, we map them to attribute- relevant sentences with a prefix-tuning-based decoder. Experiments on the three-aspect con- trol task, including sentiment, topic, and detox- ification aspects, reveal that our method outper- forms several strong baselines on attribute rele- vance and text quality and achieves the SOTA. Further analysis also supplies some explanatory support for the effectiveness of our approach1. 1 Introduction Controllable text generation is a challenging task in natural language generation, which aims to gener- ate fluent text with desired attributes. Pilot studies attempt single-aspect control by directly finetuning a conditional model (Ziegler et al., 2019; Keskar et al., 2019), or turn to methods with language mod- els fixed (Dathathri et al., 2020) due to the high cost of large-scale pre-trained language models (Brown et al., 2020a; Zhang et al., 2022). Recent works focus on a more practical setting, multi-aspect2 controllable text generation, with ex- isting approaches mainly divided into three tech- 1Our dataset and code are available at: https://github .com/HappyGu0524/MultiControl 2For example, positive is an attribute from sentiment aspect while sports is an attribute from topic aspect. Intersection Interpolation Interpolation Intersection Figure 1: Probability space of attributes. Orange back- ground denotes the estimated distribution over natural language. Blue and green areas represent distributions over sentences containing attributes from two different aspects, respectively. The darker region means a higher probability in the space. The shaded are distributional centers, the areas with the highest probability density. nical routes: weighted decoding (Dathathri et al., 2020; Krause et al., 2021), multi-objective opti- mization (Kumar et al., 2021; Mireshghallah et al., 2022), and prefix-tuning (Qian et al., 2022), which explore ways to combine controllers learned from single-aspect and apply them to a fixed language model yet suffering from attribute degeneration caused by the mutual interference of controllers. We provide a distributional perspective to ob- serve and alleviate this problem. In the current text generation paradigm, a language model forms an estimated distribution over sentences with train- ing data amounted to sampling from natural lan- guage distribution (Pillutla et al., 2021). For single- aspect control, these methods train a classifier …

Content truncated to 500 words