CLUE: A Chinese Language Understanding Evaluation Benchmark

Proceedings of the 28th International Conference on Computational Linguistics, pages 465–479 Barcelona, Spain (Online), December 8-13, 2020 465 LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization Nurul Lubis, Christian Geishauser, Michael Heck, Hsien-chin Lin, Marco Moresi, Carel van Niekerk and Milica Gaˇsi´c Heinrich Heine University D¨usseldorf, Germany {lubis,geishaus,heckmi,linh,moresi,niekerk,gasic}@hhu.de Abstract Reinforcement learning (RL) can enable task-oriented dialogue systems to steer the conversation towards successful task completion. In an end-to-end setting, a response can be constructed in a word-level sequential decision making process with the entire system vocabulary as action space. Policies trained in such a fashion do not require expert-defined action spaces, but they have to deal with large action spaces and long trajectories, making RL impractical. Using the latent space of a variational model as action space alleviates this problem. However, current approaches use an uninformed prior for training and optimize the latent distribution solely on the context. It is therefore unclear whether the latent representation truly encodes the characteristics of different actions. In this paper, we explore three ways of leveraging an auxiliary task to shape the latent variable distribution: via pre-training, to obtain an informed prior, and via multitask learning. We choose response auto-encoding as the auxiliary task, as this captures the generative factors of dialogue responses while requiring low computational cost and neither additional data nor labels. Our approach yields a more action-characterized latent representations which support end-to-end dialogue policy optimization and achieves state-of-the-art success rates. These results warrant a more wide-spread use of RL in end-to-end dialogue models. 1 Introduction With the rise of personal assistants, task-oriented dialogue systems have received a surge in popularity and acceptance. Task-oriented dialogue systems are characterized by a user goal which motivates the interaction, e.g. booking a hotel, searching for a restaurant, or calling a taxi. The dialogue agent is considered successful if it is able to fulfill the user goal by the end of the interaction. Traditionally, a dialogue system is built using the divide and conquer approach, resulting in multiple modules that to- gether form a dialogue system pipeline: a natural language understanding (NLU) module, a dialogue state tracker (DST), a dialogue policy, and a natural language generation (NLG) module. Each module has well-defined input and output, and can be trained using machine learning (Young et al., 2010; Thom- son and Young, 2010; Williams, 2006; Henderson et al., 2013) provided that adequately labeled data is available. However, there is a loss of information between the modules. The availability of powerful deep learning methods has recently led to a surge in end-to-end training approaches (Bordes and Weston, 2017; Wen et al., 2017; Madotto et al., 2018), which aim to map user utterances directly to responses in a sequence-to-sequence fashion. Two fundamental properties of task-oriented systems are the ability to remember everything that is important from the conversation so far – tracking, and the ability to produce a response that steers the conversation towards successful task completion – planning. Within the modular approaches the role of tracking …

Content truncated to 500 words