HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data

Findings of the Association for Computational Linguistics: ACL 2022, pages 481 - 492 May 22-27, 2022 c⃝2022 Association for Computational Linguistics HYBRIDIALOGUE: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data Kai Nakamura1, Sharon Levy2, Yi-Lin Tuan2, Wenhu Chen3, William Yang Wang2 1 California Institute of Technology 2 University of California, Santa Barbara 3 University of Waterloo, Vector Institute kai.nakamura42@gmail.com {sharonlevy,ytuan,william}@cs.ucsb.edu wenhuchen@uwaterloo.ca Abstract A pressing challenge in current dialogue sys- tems is to successfully converse with users on topics with information distributed across dif- ferent modalities. Previous work in multiturn dialogue systems has primarily focused on ei- ther text or table information. In more real- istic scenarios, having a joint understanding of both is critical as knowledge is typically distributed over both unstructured and struc- tured forms. We present a new dialogue dataset, HYBRIDIALOGUE, which consists of crowd- sourced natural conversations grounded on both Wikipedia text and tables. The conversations are created through the decomposition of com- plex multihop questions into simple, realistic multiturn dialogue interactions. We propose retrieval, system state tracking, and dialogue re- sponse generation tasks for our dataset and con- duct baseline experiments for each. Our results show that there is still ample opportunity for improvement, demonstrating the importance of building stronger dialogue systems that can reason over the complex setting of information- seeking dialogue grounded on tables and text. 1 Introduction When creating dialogue systems, researchers strive to enable fluent free-text interactions with users on a number of topics. These systems can be utilized to navigate users over the vast amount of online content to answer the user’s question. Current sys- tems may search for information within text pas- sages. However, knowledge comes in many forms other than text. The ability to understand multi- ple knowledge forms is critical in developing more general-purpose and realistic conversational mod- els. Tables often convey information that cannot be efficiently captured via text, such as structured relational representations between multiple enti- ties across different categories (Chen et al., 2019, 2020b; Herzig et al., 2020). On the other hand, text may contain more detailed information regarding a specific entity. Thus, dialogue systems must be able to effectively incorporate and reason across both modalities to yield the best performance in the real world. While there are several existing datasets tar- geted at dialogue systems (Dinan et al., 2018; Budzianowski et al., 2018; Eric et al., 2017; Zhou et al., 2018b), these are limited to either table-only or text-only information sources. As a result, cur- rent dialogue systems may fail to respond correctly in situations that require combined tabular and tex- tual knowledge. To advance the current state of dialogue systems, we create HYBRIDIALOGUE 1. Our dataset is an information-seeking dialogue dataset grounded on structured and unstructured knowledge from tables and text. HYBRIDIALOGUE, or HYDI, is con- structed by decomposing the complex and artifi- cial multihop questions in OTT-QA (Chen et al., 2020a) which may not reflect real-life queries. We transform these into a series of simple and more re- alistic intermediate …

Content truncated to 500 words