Policy Learning for Domain Selection in an Extensible Multi-domain Spoken Dialogue System

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 57–67, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Policy Learning for Domain Selection in an Extensible Multi-domain Spoken Dialogue System Zhuoran Wang Mathematical & Computer Sciences Heriot-Watt University Edinburgh, UK zhuoran.wang@hw.ac.uk Hongliang Chen, Guanchun Wang Hao Tian, Hua Wu†, Haifeng Wang Baidu Inc., Beijing, P. R. China SurnameForename@baidu.com †wu hua@baidu.com Abstract This paper proposes a Markov Decision Process and reinforcement learning based approach for domain selection in a multi- domain Spoken Dialogue System built on a distributed architecture. In the proposed framework, the domain selection prob- lem is treated as sequential planning in- stead of classification, such that confir- mation and clarification interaction mech- anisms are supported. In addition, it is shown that by using a model parameter ty- ing trick, the extensibility of the system can be preserved, where dialogue com- ponents in new domains can be easily plugged in, without re-training the domain selection policy. The experimental results based on human subjects suggest that the proposed model marginally outperforms a non-trivial baseline. 1 Introduction Due to growing demand for natural human- machine interaction, over the last decade Spo- ken Dialogue Systems (SDS) have been increas- ingly deployed in various commercial applications ranging from traditional call centre automation (e.g. AT&T “Lets Go!” bus information sys- tem (Williams et al., 2010)) to mobile personal assistants and knowledge navigators (e.g. Ap- ple’s Siri R⃝, Google NowTM, Microsoft Cortana, etc.) or voice interaction for smart household ap- pliance control (e.g. Samsung Evolution Kit for Smart TVs). Furthermore, latest progress in open- vocabulary Automatic Speech Recognition (ASR) is pushing SDS from traditional single-domain in- formation systems towards more complex multi- domain speech applications, of which typical ex- amples are those voice assistant mobile applica- tions. Recent advances in SDS have shown that sta- tistical approaches to dialogue management can result in marginal improvement in both the nat- uralness and the task success rate for domain- specific dialogues (Lemon and Pietquin, 2012; Young et al., 2013). State-of-the-art statistical SDS treat the dialogue problem as a sequential decision making process, and employ established planning models, such as Markov Decision Pro- cesses (MDPs) (Singh et al., 2002) or Partially Ob- servable Markov Decision Processes (POMDPs) (Thomson and Young, 2010; Young et al., 2010; Williams and Young, 2007), in conjunction with reinforcement learning techniques (Jurˇc´ıˇcek et al., 2011; Jurˇc´ıˇcek et al., 2012; Gaˇsi´c et al., 2013a) to seek optimal dialogue policies that maximise long-term expected (discounted) rewards and are robust to ASR errors. However, to the best of our knowledge, most of the existing multi-domain SDS in public use are rule-based (e.g. (Gruber et al., 2012; Mirkovic and Cavedon, 2006)). The application of statistical models in multi-domain dialogue systems is still preliminary. Komatani et al. (2006) and Nakano et al. (2011) utilised a distributed architecture (Lin et al., 1999) to integrate expert dialogue systems in different domains into a unified framework, where a central controller trained as a data-driven clas- …

Content truncated to 500 words