Conversational Tree Search: A New Hybrid Dialog Task

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1264–1280 May 2-6, 2023 ©2023 Association for Computational Linguistics Conversational Tree Search: A New Hybrid Dialog Task Dirk Väth Lindsey Vanderlyn University of Stuttgart, Germany {vaethdk|vanderly|thang.vu}@ims.uni-stuttgart.de Ngoc Thang Vu Abstract Conversational interfaces provide a flexible and easy way for users to seek information that may otherwise be difficult or inconvenient to obtain. However, existing interfaces gen- erally fall into one of two categories: FAQs, where users must have a concrete question in order to retrieve a general answer, or dialogs, where users must follow a predefined path but may receive a personalized answer. In this pa- per, we introduce Conversational Tree Search (CTS) as a new task that bridges the gap between FAQ-style information retrieval and task-oriented dialog, allowing domain-experts to define dialog trees which can then be con- verted to an efficient dialog policy that learns only to ask the questions necessary to navigate a user to their goal. We collect a dataset for the travel reimbursement domain and demonstrate a baseline as well as a novel deep Reinforce- ment Learning architecture for this task. Our results show that the new architecture com- bines the positive aspects of both the FAQ and dialog system used in the baseline and achieves higher goal completion while skip- ping unnecessary questions. 1 Introduction Complex processes, e.g., healthcare, insurance, or travel reimbursement, can be challenging for users to navigate. FAQ and task-oriented dialog systems can provide immediate support, which is especially helpful as questions in these areas can be time sen- sitive, e.g., whom to contact in case of a lost pass- port. To support all types of users, however, such systems must be fast to satisfy those with more experience and thorough, so that those with less ex- perience understand all steps needed to accomplish their goals. Above all, however, accuracy is critical in such susceptible domains. FAQ systems can directly answer to user queries by matching them to predefined common ques- tion/answer pairs. It enables quick access to in- formation while allowing subject experts to define Figure 1: An example of the proposed task: Slice of a dialog tree (blue/gray nodes, black edges) showing how progressively more concrete questions could be answered. Question a) guiding a user with a general goal through the tree, b) asking only at nodes that need more clarification, and c) requiring no clarification and thus receiving a direct answer. system output and ensure it is factually correct (Wu et al., 2005). However, this approach does not al- low for personalizing answers to a user’s specific case, e.g., a specific user’s per diem might depend on their destination and length of travel. Including information for all cases in one FAQ answer would make it long and hard to understand, while adding FAQs for each case, would make retrieval challeng- ing. Additionally, as retrieval accuracy could be better (Thakur et al., 2021), the top answer might be incorrect, but displaying multiple answers …

Content truncated to 500 words