Modeling Long Context for Task-Oriented Dialogue State Generation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7119–7124 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7119 Modeling Long Context for Task-Oriented Dialogue State Generation Jun Quan and Deyi Xiong∗ School of Computer Science and Technology, Soochow University, Suzhou, China terryqj0107@gmail.com, dyxiong@suda.edu.cn Abstract Based on the recently proposed transfer- able dialogue state generator (TRADE) (Wu et al., 2019) that predicts dialogue states from utterance-concatenated dialogue context, we propose a multi-task learning model with a simple yet effective utterance tagging tech- nique and a bidirectional language model as an auxiliary task for task-oriented dialogue state generation. By enabling the model to learn a better representation of the long dialogue con- text, our approaches attempt to solve the prob- lem that the performance of the baseline sig- nificantly drops when the input dialogue con- text sequence is long. In our experiments, our proposed model achieves a 7.03% relative improvement over the baseline, establishing a new state-of-the-art joint goal accuracy of 52.04% on the MultiWOZ 2.0 dataset. 1 Introduction Dialogue state tracking (DST, also known as belief tracking) predicts user’s goals in task-oriented dia- logue system, where dialogue states are normally represented in the form of a set of slot-value pairs. A variety of approaches to dialogue state tracking are devoted to dealing with two different settings: DST over a predefined domain ontology and DST with slot-value candidates from an open vocabu- lary. Most of the previous work is based on the first setting, assuming that all possible slot-value candidates are provided in a domain ontology in advance. The task of the dialogue state tracking with this setting is therefore largely simplified to score all predefined slot-value pairs and select the value with the highest score for each slot as the final prediction. Although predefined ontology-based approaches are successfully used on datasets with small ontologies, such as DSTC2 (Henderson et al., 2014) and WOZ2.0 (Wen et al., 2017), they are ∗Corresponding author quite limited in both scalability to scenarios with infinite slot values and prediction of unseen slot values. In order to address these issues of DST over pre- defined ontologies, recent efforts have been made to predict slot-value pairs in open vocabularies. Among them, TRADE (Wu et al., 2019) proposes to encode the entire dialogue context and to pre- dict the value for each slot using a copy-augmented decoder, achieving state-of-the-art results on the MultiWOZ 2.0 dataset (Budzianowski et al., 2018). As TRADE simply concatenates all the system and user utterances in previous turns into a single se- quence as the dialogue context for slot-value predic- tion, it is difficult for the model to identify whether an utterance in the dialogue context is from system or user when the concatenated sequence becomes long. We observe that the longest dialogue context after concatenation on the MultiWOZ 2.0 dataset contains 880 tokens. Our experiments also demon- strate that the longer the dialogue context sequence is, the worse TRADE performs. To deal with this problem, we propose …

Content truncated to 500 words