TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Base

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8108 - 8121 December 7-11, 2022 ©2022 Association for Computational Linguistics TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases Yiheng Shu♡∗, Zhiwei Yu♢, Yuhan Li♣∗, Börje F. Karlsson♢, Tingting Ma♠∗, Yuzhong Qu♡ and Chin-Yew Lin♢ ♡ State Key Laboratory of Novel Software Technology, Nanjing University, China; ♢ Microsoft Research; ♣ Nankai University; ♠ Harbin Institute of Technology yhshu@smail.nju.edu.cn, yzqu@nju.edu.cn, yuhanli@mail.nankai.edu.cn {zhiwei.yu,borje.karlsson,cyl}@microsoft.com Abstract Pre-trained language models (PLMs) have shown their effectiveness in multiple scenar- ios. However, KBQA remains challenging, especially regarding coverage and generaliza- tion settings. This is due to two main factors: i) understanding the semantics of both ques- tions and relevant knowledge from the KB; ii) generating executable logical forms with both semantic and syntactic correctness. In this paper, we present a new KBQA model, TIARA, which addresses those issues by ap- plying multi-grained retrieval to help the PLM focus on the most relevant KB contexts, viz., entities, exemplary logical forms, and schema items. Moreover, constrained decoding is used to control the output space and reduce genera- tion errors. Experiments over important bench- marks demonstrate the effectiveness of our ap- proach. TIARA outperforms previous SOTA, including those using PLMs or oracle entity annotations, by at least 4.1 and 1.1 F1 points on GrailQA and WebQuestionsSP, respectively. Specifically on GrailQA, TIARA outperforms previous models in all categories, with an im- provement of 4.7 F1 points in zero-shot gener- alization.1 1 Introduction Knowledge base question answering (KBQA) has established itself as an important and promising re- search area as it greatly helps the accessibility and usability of existing large-scale knowledge bases (KBs). Such KBs contain abundant facts in a struc- tured form, which can not only be accessed, but also reasoned over. KBQA bypasses the need for users to learn complex and burdensome formal query languages, and it allows users to leverage knowledge using natural language. While recent KBQA efforts have achieved interesting results, 1Code is available at https://github.com/microsoft/KC/ tree/main/papers/TIARA. *Work performed during their internships at Microsoft Research Asia. most previous studies target or assume an under- lying strong correspondence between the distribu- tions of schema items in test questions and training data, i.e., the i.i.d. assumption. However, this as- sumption does not hold, especially for large-scale KBs, which typically contain very large numbers of entities and schema items (classes and relations).2 This extensive space is a critical challenge as it both allows for a myriad of novel compositions of schema items (Keysers et al., 2020), i.e., requires compositional generalization. It also immensely in- creases the likelihood of user queries, including pre- viously unseen items or domains (Gu et al., 2021), i.e., requires strong zero-shot generalization. Meanwhile, pre-trained language models (PLMs), such as T5 (Raffel et al., 2020) and GPT-3 (Brown et al., 2020), have demonstrated notable success in many natural language processing (NLP) scenarios (Karpas et al., 2022). Many times they even show strong generalization capabilities. Inspired by the progress of PLMs on unstructured …

Content truncated to 500 words