Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog

Findings of the Association for Computational Linguistics: EACL 2023, pages 1778–1787 May 2-6, 2023 ©2023 Association for Computational Linguistics Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog Mayank Mishra IBM Research mayank.mishra1@ibm.com Danish Contractor IBM Research danish.contractor@ibm.com Dinesh Raghu IBM Research diraghu1@in.ibm.com Abstract Traditional systems designed for task oriented dialog utilize knowledge present only in struc- tured knowledge sources to generate responses. However, relevant information required to gen- erate responses may also reside in unstructured sources, such as documents. Recent state of the art models such as HyKnow (Gao et al., 2021b) and SEKNOW (Gao et al., 2021a) aimed at overcoming these challenges make limiting as- sumptions about the knowledge sources. For in- stance, these systems assume that certain types of information, such as a phone number, is al- ways present in a structured knowledge base (KB) while information about aspects such as entrance ticket prices, would always be avail- able in documents. In this paper, we create a modified version of the MutliWOZ-based dataset prepared by (Gao et al., 2021a) to demonstrate how current methods have significant degradation in per- formance when strict assumptions about the source of information are removed. Then, in line with recent work exploiting pre-trained lan- guage models, we fine-tune a BART (Lewis et al., 2020) based model using prompts (Brown et al., 2020; Sun et al., 2021) for the tasks of querying knowledge sources, as well as, for response generation, without making assumptions about the information present in each knowledge source. Through a series of experiments, we demonstrate that our model is robust to perturbations to knowledge modality (source of information), and that it can fuse information from structured as well as unstruc- tured knowledge to generate responses. 1 Introduction Most existing work on task-oriented dialog systems assumes that the knowledge required for complet- ing a task (eg: booking a restaurant reservation), resides in structured knowledge sources. Thus, typ- ical task-oriented dialog systems require generating a belief state, that can be used to query a knowledge base to fetch entity results; these results are then used to generate responses. Recognizing that infor- mation is not always present in structured resources, recently methods that can additionally use unstruc- tured knowledge (eg: document collections), have also been developed (Kim et al., 2020; Gao et al., 2021a). However, current state-of-the-art models designed for such tasks make limiting assumptions about the nature of knowledge sources, that make them unsuitable for use in real-world settings. Limitations of existing methods: First, current task-oriented dialog systems designed to reason over hybrid knowledge sources assume that a knowledge base and the unstructured knowledge source encode separate pieces of information about entities (eg: the zip-code is always in structured knowledge, ticket prices are always available in unstructured text) (Kim et al., 2020; Zhang et al., 2021). This is not reflective of real-world knowl- edge, where independent information systems are often fused to enable applications. Second, existing systems are trained to learn the source of different pieces of information, thus, making them unsuitable for …

Content truncated to 500 words