Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4078–4088 August 1–6, 2021. ©2021 Association for Computational Linguistics 4078 Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering Alexander Hanbo Li, Patrick Ng, Peng Xu, Henghui Zhu, Zhiguo Wang, Bing Xiang AWS AI Labs, Amazon {hanboli, patricng, pengx, henghui, zhiguow, bxiang}@amazon.com Abstract The current state-of-the-art generative models for open-domain question answering (ODQA) have focused on generating direct answers from unstructured textual information. How- ever, a large amount of world’s knowledge is stored in structured databases, and need to be accessed using query languages such as SQL. Furthermore, query languages can an- swer questions that require complex reason- ing, as well as offering full explainability. In this paper, we propose a hybrid framework that takes both textual and tabular evidence as input and generates either direct answers or SQL queries depending on which form could better answer the question. The gen- erated SQL queries can then be executed on the associated databases to obtain the final an- swers. To the best of our knowledge, this is the first paper that applies Text2SQL to ODQA tasks. Empirically, we demonstrate that on several ODQA datasets, the hybrid methods consistently outperforms the baseline models that only take homogeneous input by a large margin. Specifically we achieve state-of-the- art performance on OpenSQuAD dataset us- ing a T5-base model. In a detailed analysis, we demonstrate that the being able to gener- ate structural SQL queries can always bring gains, especially for those questions that re- quires complex reasoning. 1 Introduction Open-domain question answering (ODQA) is a task to answer factoid questions without a pre- specified domain. Recently, generative models (Roberts et al., 2020; Lewis et al., 2020; Min et al., 2020; Izacard and Grave, 2020) have achieved the state-of-the-art performance on many ODQA tasks. These approaches all share the common pipeline where the first stage is retrieving evidence from the free-form text in Wikipedia. However, a large amount of world’s knowledge is not stored as plain text but in structured databases, and need to be accessed using query languages such as SQL. Fur- thermore, query languages can answer questions that require complex reasoning, as well as offer- ing full explainability. In practice, an ideal ODQA model should be able to retrieve evidence from both unstructured textual and structured tabular informa- tion sources, as some questions are better answered by tabular evidence from databases. For example, the current state-of-the-art ODQA models struggle on questions that involve aggregation operations such as counting or averaging. One line of research on accessing databases, al- though not open domain, is translating natural lan- guage questions into SQL queries (Zhong et al., 2017; Xu et al., 2017; Yu et al., 2018c; Guo et al., 2019; Wang et al., 2018a, 2020; Yu et al., 2018a; Guo and Gao, 2019; Choi et al., 2020). These methods all rely on knowing the associated table for each question …

Content truncated to 500 words