Structured Attention for Unsupervised Dialogue Structure Induction

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1889–1899, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 1889 Structured Attention for Unsupervised Dialogue Structure Induction Liang Qiu1, Yizhou Zhao1, Weiyan Shi2, Yuan Liang3, Feng Shi1, Tao Yuan1, Zhou Yu2, Song-Chun Zhu1 1UCLA Center for Vision, Cognition, Learning, and Autonomy 2University of California, Davis 3University of California, Los Angeles {liangqiu,yizhouzhao,liangyuandg,shifeng,taoyuan}@ucla.edu {wyshi,joyu}@ucdavis.edu sczhu@stat.ucla.edu Abstract Inducing a meaningful structural representa- tion from one or a set of dialogues is a cru- cial but challenging task in computational lin- guistics. Advancement made in this area is critical for dialogue system design and dis- course analysis. It can also be extended to solve grammatical inference. In this work, we propose to incorporate structured atten- tion layers into a Variational Recurrent Neu- ral Network (VRNN) model with discrete la- tent states to learn dialogue structure in an unsupervised fashion. Compared to a vanilla VRNN, structured attention enables a model to focus on different parts of the source sen- tence embeddings while enforcing a structural inductive bias. Experiments show that on two-party dialogue datasets, VRNN with struc- tured attention learns semantic structures that are similar to templates used to generate this dialogue corpus. While on multi-party dia- logue datasets, our model learns an interac- tive structure demonstrating its capability of distinguishing speakers or addresses, automat- ically disentangling dialogues without explicit human annotation.1 1 Introduction Grammatical induction for capturing a structural representation of knowledge has been studied for some time (De la Higuera, 2010). Given the achievement in related areas like learning Hid- den Markov acoustic models in speech recognition (Bahl et al., 1986) and sentence dependency pars- ing in language understanding (Covington, 2001), our work aims to explore a more sophisticated topic: learning structures in dialogues. Figure 1 shows the underlying semantic structure of conver- sations about bus information request from Sim- Dial dataset (Zhao and Eskenazi, 2018), with one 1The code is released at https://github.com/ Liang-Qiu/SVRNN-dialogues. example dialogue as shown in Table 1. Another interesting type of dialogue structure is the inter- active structure in multi-party dialogues. Figure 2 illustrates the interactive structure we learned from a dialogue sample in Ubuntu Chat Corpus (Lowe et al., 2015). Each node represents an utterance from different speakers in the dialogue with darker linkages represent stronger dependency relations between utterances. When speaker/addressee in- formation is unavailable in the corpus, learning such a structure allows disentangling the conver- sation (Serban and Pineau, 2015) and estimating the speaker labels. Discovering dialogue structures is crucial for various areas in computational lin- guistics, such as dialogue system building (Young, 2006), discourse analysis (Grosz and Sidner, 1986), and dialogue summarization (Murray et al., 2005; Liu et al., 2010). Through looking into this topic, we can further improve the capability of machines to learn more generalized, interpretable knowledge representation from data. greeting request a bus request #from_loc inform #from_loc request #to_loc inform #to_loc request #datetime inform #datetime 0.49 inform default request #duration inform default request #arrival inform arrival goodbye inform …

Content truncated to 500 words