Hidden Softmax Sequence Model for Dialogue Structure Analysis

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2063–2072, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Hidden Softmax Sequence Model for Dialogue Structure Analysis Zhiyang He1, Xien Liu2, Ping Lv2, Ji Wu1 1Department of Electronic Engineering, Tsinghua University, Beijing, China 2Tsinghua-iFlytek Joint Laboratory for Speech Technology, Beijing, China {zyhe ts, xeliu, luping ts, wuji ee}@mail.tsinghua.edu.cn Abstract We propose a new unsupervised learning model, hidden softmax sequence model (HSSM), based on Boltzmann machine for dialogue structure analysis. The model employs three types of units in the hidden layer to discovery dialogue latent struc- tures: softmax units which represent latent states of utterances; binary units which represent latent topics specified by dia- logues; and a binary unit that represents the global general topic shared across the whole dialogue corpus. In addition, the model contains extra connections between adjacent hidden softmax units to formu- late the dependency between latent states. Two different kinds of real world dialogue corpora, Twitter-Post and AirTicketBook- ing, are utilized for extensive comparing experiments, and the results illustrate that the proposed model outperforms sate-of- the-art popular approaches. 1 Introduction Dialogue structure analysis is an important and fundamental task in the natural language process- ing domain. The technology provides essential clues for solving real-world problems, such as pro- ducing dialogue summaries (Murray et al., 2006; Liu et al., 2010), controlling conversational agents (Wilks, 2006), and designing interactive dialogue systems (Young, 2006; Allen et al., 2007) etc. The study of modeling dialogues always assumes that for each dialogue there exists an unique latent structure (namely dialogue structure), which con- sists of a series of latent states.1 1Also called dialogue acts or speech acts in some past work. In this paper, for simplicity we will only use the term “latent state” to describe the sequential dialogue structure. Some past works mainly rely on supervised or semi-supervised learning, which always involve extensive human efforts to manually construct la- tent state inventory and to label training samples. Cohen et al. (2004) developed an inventory of la- tent states specific to E-mail in an office domain by inspecting a large corpus of e-mail. Jeong et al. (2009) employed semi-supervised learning to transfer latent states from labeled speech corpora to the Internet media and e-mail. Involving exten- sive human efforts constrains scaling the training sample size (which is essential to supervised learn- ing) and application domains. In recent years, there has been some work on modeling dialogues with unsupervised learn- ing methods which operate only on unlabeled ob- served data. Crook et al. (2009) employed Dirich- let process mixture clustering models to recog- nize latent states for each utterance in dialogues from a travel-planning domain, but they do not inspect dialogues’ sequential structure. Choti- mongkol (2008) proposed a hidden Markov model (HMM) based dialogue analysis model to study structures of task-oriented conversations from in- domain dialogue corpus. More recently, Ritter et al. (2010) extended the HMM based conversa- tion model by introducing additional word sources for topic learning …

Content truncated to 500 words