Neural Response Generation with Meta-words

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5416–5426 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5416 Neural Response Generation with Meta-Words Can Xu1, Wei Wu1∗, Chongyang Tao2, Huang Hu1, Matt Schuerman3, and Ying Wang3 1Microsoft Corporation, Beijing, China 2Institute of Computer Science and Technology, Peking University, Beijing, China 3Microsoft Corporation, Redmond, Washington 1,3{caxu,wuwei,huahu,matthes,Ying.Wang}@microsoft.com 2chongyangtao@pku.edu.cn Abstract We present open domain response generation with meta-words. A meta-word is a structured record that describes various attributes of a re- sponse, and thus allows us to explicitly model the one-to-many relationship within open do- main dialogues and perform response genera- tion in an explainable and controllable man- ner. To incorporate meta-words into genera- tion, we enhance the sequence-to-sequence ar- chitecture with a goal tracking memory net- work that formalizes meta-word expression as a goal and manages the generation process to achieve the goal with a state memory panel and a state controller. Experimental results on two large-scale datasets indicate that our model can significantly outperform several state-of- the-art generation models in terms of response relevance, response diversity, accuracy of one- to-many modeling, accuracy of meta-word ex- pression, and human evaluation. 1 Introduction Human-machine conversation is a fundamental problem in NLP. Traditional research focuses on building task-oriented dialog systems (Young et al., 2013) to achieve specific user goals such as restaurant reservation through limited turns of dia- logues within specific domains. Recently, building a chatbot for open domain conversation (Vinyals and Le, 2015) has attracted increasing attention, not only owing to the availability of large amount of human-human conversation data on internet, but also because of the success of such systems in real products such as the social bot XiaoIce (Shum et al., 2018) from Microsoft. A common approach to implementing a chat- bot is to learn a response generation model within an encoder-decoder framework (Vinyals and Le, ∗Corresponding author. Message: last week I have a nice trip to New York! Meta-word: Act: yes-no question | Len: 8 | Copy: true | Utts: false | Spe: medium Response 1: Is New York more expensive than California? Meta-word: Act: wh-question | Len: 17 | Copy: false | Utts: true | Spe: high Response 2: Cool, sounds great! What is the tallest building in this city, Chrysler building? Meta-word: Act: statement | Len: 13 | Copy: false | Utts: true | Spe: low Response 3: I don’t know what you are talking about. But it seems good. Table 1: An example of response generation with meta- words. The underlined word means it is copied from the message, and the word in bold means it corre- sponds to high specificity. 2015; Shang et al., 2015). Although the architec- ture can naturally model the correspondence be- tween a message and a response, and is easy to ex- tend to handle conversation history (Serban et al., 2016; Xing et al., 2018) and various constraints (Li et al., 2016; Zhou et al., 2018), it is notori- ous for generating …

Content truncated to 500 words