Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3696–3709 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3696 Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention Wenhu Chen†, Jianshu Chen‡, Pengda Qin¶, Xifeng Yan† and William Yang Wang† †University of California, Santa Barbara, CA, USA ‡Tencent AI Lab, Bellevue, WA, USA ¶Beijing University of Posts and Telecommunications, China {wenhuchen,xyan,william}@cs.ucsb.edu jianshuchen@tencent.com qinpengda@bupt.edu.cn Abstract Semantically controlled neural response gen- eration on limited-domain has achieved great performance. However, moving towards multi-domain large-scale scenarios are shown to be difficult because the possible combina- tions of semantic inputs grow exponentially with the number of domains. To alleviate such scalability issue, we exploit the structure of dialog acts to build a multi-layer hierarchical graph, where each act is represented as a root- to-leaf route on the graph. Then, we incor- porate such graph structure prior as an induc- tive bias to build a hierarchical disentangled self-attention network, where we disentangle attention heads to model designated nodes on the dialog act graph. By activating differ- ent (disentangled) heads at each layer, com- binatorially many dialog act semantics can be modeled to control the neural response gener- ation. On the large-scale Multi-Domain-WOZ dataset, our model can yield a significant im- provement over the baselines on various auto- matic and human evaluation metrics. 1 Introduction Conversational artificial intelligence (Young et al., 2013) is one of the critical milestones in artifi- cial intelligence. Recently, there have been in- creasing interests in industrial companies to build task-oriented conversational agents (Wen et al., 2017; Li et al., 2017; Rojas-Barahona et al., 2017) to solve pre-defined tasks such as restaurant or flight bookings, etc (see Figure 1 for an exam- ple dialog from MultiWOZ (Budzianowski et al., 2018)). Traditional agents are built based on slot- filling techniques, which requires significant hu- man handcraft efforts. And it is hard to gener- ate naturally sounding utterances in a generaliz- able and scalable manner. Therefore, different semantically controlled neural language genera- tion models have been developed (Wen et al., 2015, 2016a,b; Dusek and Jurc´ıcek, 2016) to re- place the traditional systems, where an explicit se- mantic representation (dialog act) are used to in- fluence the RNN generation. The canonical ap- proach is proposed in (Wen et al., 2015) to en- code each individual dialog act as a unique vec- tor and use it as an extra input feature into the cell of long short-term memory (LSTM) to influence the generation. As pointed in (Wen et al., 2016b), these models though achieving good performance on limited domains, suffer from scalability prob- lem as the possible dialog acts grow combinatori- ally with the number of domains. In order to alleviate such issue, we propose a hierarchical graph representation by leveraging the structural property of dialog acts. Specifi- cally, we first build a multi-layer tree to represent the entire dialog act space based on their inter- relationships. Then, we merge the tree nodes with the same semantic meaning …

Content truncated to 500 words