Structuring Latent Spaces for Stylized Response Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1814–1823, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1814 Structuring Latent Spaces for Stylized Response Generation Xiang Gao Yizhe Zhang Sungjin Lee ∗ Michel Galley Chris Brockett Jianfeng Gao Bill Dolan Microsoft Research, Redmond, WA, USA {xiag,billdol}@microsoft.com Abstract Generating responses in a targeted style is a useful yet challenging task, especially in the absence of parallel data. With limited data, existing methods tend to generate re- sponses that are either less stylized or less context-relevant. We propose STYLEFUSION, which bridges conversation modeling and non- parallel style transfer by sharing a structured latent space. This structure allows the system to generate stylized relevant responses by sam- pling in the neighborhood of the conversation model prediction, and continuously control the style level. We demonstrate this method us- ing dialogues from Reddit data and two sets of sentences with distinct styles (arXiv and Sher- lock Holmes novels). Automatic and human evaluation show that, without sacrificing ap- propriateness, the system generates responses of the targeted style and outperforms competi- tive baselines. 1 1 Introduction A social chatbot designed to establish long-term emotional connections with users must generate responses that not only match the content of user input and context, but also do so in a desired tar- get style (Zhou et al., 2018; Li et al., 2016b; Luan et al., 2016; Gao et al., 2019a). A conversational agent that speaks in a polite, professional tone is likely to facilitate service in customer relationship scenarios; likewise, an agent that sounds like an cartoon character or a superhero can be more en- gaging in a theme park. The master of response style is also an important step towards human- like chatbots. As highlighted in social psychology studies (Niederhoffer and Pennebaker, 2002a,b), when two people are talking, they tend to match ∗Now at Alexa AI, Amazon. 1An implementation of our model and the scripts to gener- ate the datasets are available at https://github.com/ golsun/StyleFusion. Figure 1: STYLEFUSION helps conversational model to distill style from non-conversational, non-parallel sentences by mapping them to points surrounding the related conversations in the structured latent space. Di- rection and distance from the model prediction (center black dot) roughly correspond to contents and style in- tensity, respectively, illustrated by examples taken from Table 2. linguistic style of each other, sometime even re- gardless of their intentions. Achieving this level of performance, however, is challenging. Lack- ing parallel data in different conversational styles, researchers often resort to what we will term style datasets that are in non-conversational format (e.g. news, novels, blogs). Since the contents and for- mats of these are quite different from conversa- tion data, existing approaches tend to generate re- sponses that are either less style-specific (Luan et al., 2017) or less context-relevant (Niu and Bansal, 2018). We suggest that this trade-off between appropri- ateness and style stems from profound differences 1815 between conversation …

Content truncated to 500 words