Integrated Learning of Dialog Strategies and Semantic Parsing

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 547–557, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Integrated Learning of Dialog Strategies and Semantic Parsing Aishwarya Padmakumar Jesse Thomason Department of Computer Science, University of Texas at Austin {aish, jesse, mooney}@cs.utexas.edu Raymond J. Mooney Abstract Natural language understanding and dia- log management are two integral compo- nents of interactive dialog systems. Pre- vious research has used machine learning techniques to individually optimize these components, with different forms of direct and indirect supervision. We present an approach to integrate the learning of both a dialog strategy using reinforcement learn- ing, and a semantic parser for robust nat- ural language understanding, using only natural dialog interaction for supervision. Experimental results on a simulated task of robot instruction demonstrate that joint learning of both components improves di- alog performance over learning either of these components alone. 1 Introduction Natural language understanding and dialog man- agement are two integral components of a dialog system. Current research typically deals with opti- mizing only one of these components. We present an approach to integrate the learning of both a di- alog strategy using reinforcement learning, and a semantic parser for robust natural language under- standing, using only natural dialog interaction for supervision. Research in dialog systems has primarily been focused on the problems of accurate dialog state tracking and learning a policy for the dialog sys- tem to respond appropriately in various scenarios. Dialogs are typically modeled using Partially Ob- servable Markov Decision Processes (POMDPs), and various reinforcement learning algorithms have been proposed and evaluated for the task of learning optimal policies over these representa- tions to accomplish user goals using as short and natural a dialog as possible (Gaˇsi´c and Young, 2014; Pietquin et al., 2011; Young et al., 2013). However, such systems typically assume a fixed language understanding component that is avail- able a priori. Semantic parsing is the task of mapping natural language to a formal meaning representation. It has the potential to allow for more robust mapping of free-form natural language to a representation that can be used to interpret user intentions and track dialog state. This is done by leveraging the compositionality of meaning inherent in language. Prior work has shown that a semantic parser, incre- mentally updated from conversations, is helpful in dialogs for communicating commands to a mobile robot (Thomason et al., 2015). We show that in- cremental learning of a POMDP-based dialog pol- icy allows for further improvement in dialog suc- cess. A major challenge with combining the above parser and dialog policy learning techniques is that reinforcement learning (RL) algorithms as- sume that the dialog agent is operating in a sta- tionary environment. This assumption is violated when the parser is updated between conversations. For example, the improved semantic parser may be able to extract more information from a re- sponse to a question, which the old parser could not parse. So the …

Content truncated to 500 words