Meta-Reinforced Multi-Domain State Generator for Dialogue Systems

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7109–7118 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7109 Meta-Reinforced Multi-Domain State Generator for Dialogue Systems Yi Huang, Junlan Feng, Min Hu, Xiaoting Wu, Xiaoyu Du, Shuo Ma JIUTIAN Team, China Mobile Research {huangyi,fengjunlan,humin}@chinamobile.com {wuxiaoting,duxiaoyu,mashuo}@chinamobile.com Abstract A Dialogue State Tracker (DST) is a core com- ponent of a modular task-oriented dialogue system. Tremendous progress has been made in recent years. However, the major chal- lenges remain. The state-of-the-art accuracy for DST is below 50% for a multi-domain di- alogue task. A learnable DST for any new domain requires a large amount of labeled in- domain data and training from scratch. In this paper, we propose a Meta-Reinforced Multi- Domain State Generator (MERET). Our first contribution is to improve the DST accuracy. We enhance a neural model based DST gen- erator with a reward manager, which is built on policy gradient reinforcement learning (R- L) to fine-tune the generator. With this change, we are able to improve the joint accuracy of DST from 48.79% to 50.91% on the Multi- WOZ corpus. Second, we explore to train a DST meta-learning model with a few domains as source domains and a new domain as target domain. We apply the model-agnostic meta- learning (MAML) algorithm to DST and the obtained meta-learning model is used for new domain adaptation. Our experimental results show this solution is able to outperform the tra- ditional training approach with extremely less training data in target domain. 1 Introduction A Dialogue State Tracker (DST) is a core compo- nent of a modular task-oriented dialogue system (Young et al., 2013). For each dialogue turn, a DST module takes the user utterance and the dialogue history as input, and outputs a belief estimate of the dialogue state. The dialogue state as of today is simplified as a set of requests and goals, both of which are represented as (slot, value) pairs such as (area, centre), (food, Chinese) for a user request I’m looking for a Chinese restaurant in the centre of the city. A highly accurate DST is crucial to ensure ���� ������ ��������������������� ��������������������������moderate price��������south. ���������������������������������������� �������������������������������������������� �������������������������������������������������� ����������������������������������������������� �������������������������������������������������� ����������������������������� ��������������������������������������������������� ��������������������� ����������������������������������������������������������� ������������������������������������������� ����������������������������������������������������� ����������������������������the hotel ��������� ����������������������� ��������������������������������������������������������������������� ��������������������������������������������������������������� ���������� Figure 1: An example of dialogue state tracking pro- cess for booking a hotel, looking for an attraction and booking a taxi between them. Each turn contains a us- er utterance (grey) and a system utterance (blue). The dialogue state tracker (yellow) tracks all the (domain, slot, value) until the current turn. Blue color texts indi- cate mentions of slot values appeared at that turn. Best viewed in color. the quality and smoothness of a human-machine dialogue. Budzianowski et al. (2018) recently introduced a multi-domain dialogue dataset Multi-domain Wizard-of-Oz (MultiWOZ), which is more than one order of magnitude larger than all previous annotated task-oriented corpora with around 10k dialogues and involves more than 7 domains. A domain of a task-oriented system is …

Content truncated to 500 words