End-to-End Learning of Flowchart Grounded Task-Oriented Dialogs

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4348–4366 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4348 End-to-End Learning of Flowchart Grounded Task-Oriented Dialogs Dinesh Raghu ∗ † 1 2, Shantanu Agarwal ∗ 1, Sachindra Joshi 2 and Mausam 1 1 Indian Institute of Technology, New Delhi, India 2 IBM Research, New Delhi, India diraghu1@in.ibm.com, a.shantanu08@gmail.com, jsachind@in.ibm.com, mausam@cse.iitd.ac.in Abstract We propose a novel problem within end-to- end learning of task oriented dialogs (TOD), in which the dialog system mimics a trou- bleshooting agent who helps a user by diag- nosing their problem (e.g., car not starting). Such dialogs are grounded in domain-specific flowcharts, which the agent is supposed to follow during the conversation. Our task ex- poses novel technical challenges for neural TOD, such as grounding an utterance to the flowchart without explicit annotation, refer- ring to additional manual pages when user asks a clarification question, and ability to follow unseen flowcharts at test time. We release a dataset (FLODIAL) consisting of 2,738 di- alogs grounded on 12 different troubleshoot- ing flowcharts. We also design a neural model, FLONET, which uses a retrieval-augmented generation architecture to train the dialog agent. Our experiments find that FLONET can do zero-shot transfer to unseen flowcharts, and sets a strong baseline for future research. 1 Introduction Task oriented dialog (TOD) systems (Bordes and Weston, 2017) converse with users to help them with specific tasks such as calendar enquiry (Eric et al., 2017), restaurant reservation (Henderson et al., 2014), and tourist package recommenda- tion (El Asri et al., 2017). These dialog systems (e.g., restaurant reservation system) are trained us- ing past human-to-human dialogs and associated knowledge sources (e.g., a KB of restaurants). Most existing TOD systems are conversational recommender systems that gather user require- ments in the form of attributes (such as cuisine, location), query a KB and generate recommenda- tions based on the retrieved results (e.g, restaurant, its phone number). While there have been recent *D. Raghu and S.Agarwal contributed equally to this work. †D. Raghu is an employee at IBM Research. This work was carried out as part of PhD research at IIT Delhi. Utterance Type % [T1] Problem Description 6.09 [T2] Grounded on Flowchart 56.76 [T3] Grounded on Supplementary Knowledge 15.23 [T4] Chit-Chat 5.23 [T5] Conversation Markers 7.62 [T6] Hold Request 2.38 [T7] Reconfirmation 0.95 [T8] Dialog Closing 5.71 Table 1: Type of utterances and their proportions in a real-world troubleshooting dialog dataset. efforts (Feng et al., 2020; Kim et al., 2020) to study non-recommendation TOD, several important tasks such as troubleshooting are still unexplored. Troubleshooting is a common task handled by customer support agents. It involves understanding a user’s problem, narrowing down the root cause and providing a solution. Figure 1 shows an ex- ample dialog between an agent and a user trou- bleshooting a car problem. Support agents typi- cally follow a flowchart (utterances A1, A4 in our example) to diagnose user problems, but may re- fer to supplementary knowledge sources like FAQs (A3), …

Content truncated to 500 words