Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 605–614, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models Yi Luan†∗ Chris Brockett‡ Bill Dolan‡ Jianfeng Gao‡ Michel Galley‡ †Department of Electrical Engineering, University of Washington ‡Microsoft Research luanyi@uw.edu, {chrisbkt,billdol,jfgao,mgalley}@microsoft.com Abstract Building a persona-based conversation agent is challenging owing to the lack of large amounts of speaker-specific conver- sation data for model training. This pa- per addresses the problem by proposing a multi-task learning approach to train- ing neural conversation models that lever- ages both conversation data across speak- ers and other types of data pertaining to the speaker and speaker roles to be modeled. Experiments show that our approach leads to significant improvements over baseline model quality, generating responses that capture more precisely speakers’ traits and speaking styles. The model offers the ben- efits of being algorithmically simple and easy to implement, and not relying on large quantities of data representing spe- cific individual speakers. 1 Introduction Conversational engines are key components of intelligent “personal assistants” such as Apple’s Siri and Amazon’s Alexa. These assistants can perform simple tasks, answer questions, pro- vide recommendations, and even engage in chit- chats (De Mori et al., 2008; Chen et al., 2015, 2016). The emergence of these agents has been paralleled by burgeoning interest in train- ing natural-sounding dialog systems from conver- sational exchanges between humans (Ritter et al., 2011; Sordoni et al., 2015; Luan et al., 2014, 2015; Vinyals and Le, 2015). A major challenge for data-driven systems is how to generate output that corresponds to specific traits that the agent needs to adopt, as they tend to generate “consensus” re- * This work was performed at Microsoft. User input: I am getting a loop back to login page. Baseline model: Ah, ok. Thanks for the info. Our model: I’m sorry to hear that. Have you tried clearing your cache and cookies? Figure 1: Existing neural conversational models (baseline) tend to produce generic responses. The system presented in this paper better represents the speaker role (support person), domain of expertise (technical), and speaking style (courteous). sponses that are often commonplace and uninter- esting (Li et al., 2016a; Shao et al., 2017). This is illustrated in Fig. 1, where the output of a standard Sequence-to-Sequence conversation model is contrasted with that of the best system presented in this work. The baseline system gen- erates a desultory answer that offers no useful in- formation and is unlikely to inspire user confi- dence. The output of the second system, how- ever, strongly reflects the agent’s role in providing technical support. It not only evidences domain knowledge, but also manifests the professional po- liteness associated with a speaker in that role. The challenge for neural conversation systems, then, is that an agent needs to exhibit identifi- able role-specific characteristics (a ‘persona’). In practice, however, the conversational data needed to train such systems may be scarce or unavail- able in many domains. …

Content truncated to 500 words