Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues

Findings of the Association for Computational Linguistics: EACL 2023, pages 432–441 May 2-6, 2023 ©2023 Association for Computational Linguistics Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues Thibault Cordier1,2 and Tanguy Urvoy2 and Fabrice Lefèvre1 and Lina M. Rojas-Barahona2 1LIA - Avignon University, France 2Orange Innovation, Lannion, France thibault.cordier@alumni.univ-avignon.fr fabrice.lefevre@univ-avignon.fr {linamaria.rojasbarahona, tanguy.urvoy}@orange.com Abstract Reinforcement learning has been widely adopted to model dialogue managers in task- oriented dialogues. However, the user simula- tor provided by state-of-the-art dialogue frame- works are only rough approximations of human behaviour. The ability to learn from a small number of human interactions is hence crucial, especially on multi-domain and multi-task en- vironments where the action space is large. We therefore propose to use structured policies to improve sample efficiency when learning on these kinds of environments. We also evaluate the impact of learning from human vs simu- lated experts. Among the different levels of structure that we tested, the graph neural net- works (GNNs) show a remarkable superiority by reaching a success rate above 80% with only 50 dialogues, when learning from simu- lated experts. They also show superiority when learning from human experts, although a perfor- mance drop was observed, indicating a possible difficulty in capturing the variability of human strategies. We therefore suggest to concentrate future research efforts on bridging the gap be- tween human data, simulators and automatic evaluators in dialogue frameworks. 1 Introduction Multi-domain multi-task dialogue systems are de- signed to complete specific tasks in distinct do- mains such as finding and booking a hotel or a restaurant (Zhu et al., 2020). A domain is formally defined as a list of slots with their valid values. The most common task, the information-seeking task, is usually modelled as a slot-filling data-query prob- lem in which the system requests constraints to the user and proposes items that fulfil those constraints. The design of a dialogue manager (DMs) is costly: hand-crafted policies require a lot of en- gineering, pure supervised learning (or behaviour cloning) requires a lot of expert demonstrations, and pure reinforcement learning requires a lot of user interactions to converge. The simulators pro- vided with frameworks, such as PYDIAL (Ultes et al., 2017) or CONVLAB (Zhu et al., 2020), are only rough approximations of human behaviour and the ability to learn from a small number of hu- man interactions remains crucial. This is especially true on multi-domain and multi-task environments where the action space is large (Gao et al., 2018). A popular approach to reduce these costs is to wire some knowledge about the problem into the policy model, namely: few shot learning (Wang et al., 2020). In particular, structured policies like graph neural networks (GNNs) are known to be well suited to handle a variable number of slots and domains for the information-seeking task (Chen et al. 2018; Chen et al. 2020). In this paper, we explore structured policies based on GNN. A graph in a GNN is fully connected and directed. Each node represents a sub-policy associated with a slot, …

Content truncated to 500 words