RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6032 - 6043 May 22-27, 2022 c⃝2022 Association for Computational Linguistics RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering Xi Ye♦∗ Semih Yavuz♠ Kazuma Hashimoto♠ Yingbo Zhou♠ Caiming Xiong♠ ♦ Department of Computer Science, The University of Texas at Austin ♠ Salesforce Research ♦xiye@cs.utexas.edu ♠{syavuz,k.hashimoto,yingbo.zhou,cxiong}@salesforce.com Abstract Existing KBQA approaches, despite achieving strong performance on i.i.d. test data, often struggle in generalizing to questions involv- ing unseen KB schema items. Prior ranking- based approaches have shown some success in generalization, but suffer from the coverage issue. We present RnG-KBQA, a Rank-and- Generate approach for KBQA, which reme- dies the coverage issue with a generation model while preserving a strong generaliza- tion capability. Our approach first uses a con- trastive ranker to rank a set of candidate logical forms obtained by searching over the knowl- edge graph. It then introduces a tailored gener- ation model conditioned on the question and the top-ranked candidates to compose the fi- nal logical form. We achieve new state-of- the-art results on GRAILQA and WEBQSP datasets. In particular, our method surpasses the prior state-of-the-art by a large margin on the GRAILQA leaderboard. In addition, RnG- KBQA outperforms all prior approaches on the popular WEBQSP benchmark, even in- cluding the ones that use the oracle entity link- ing. The experimental results demonstrate the effectiveness of the interplay between ranking and generation, which leads to the superior per- formance of our proposed approach across all settings with especially strong improvements in zero-shot generalization.1 1 Introduction Modern knowledge bases (KB) are reliable sources of a huge amount of world knowledge but can be difficult to interact with since they are extremely large in scale and require specific query languages (e.g., Sparql) to access. Question Answering over Knowledge Base (KBQA) serves as a user-friendly way to query over KBs and has garnered increasing attention (Berant et al., 2013; Cai and Yates, 2013). Recent research has attempted to build systems ∗ Work done during internship at Salesforce Research. 1Code available at https://github.com/salesforce/rng-kbqa. (AND music.album (JOIN album.artist samuel_ramey)) (JOIN (R recording.length) (JOIN recording.artist samuel_ramey)) (AND music.recording (JOIN (R artist.track) samuel_ramey)) Enumerated Candidates ... what is the shortest recording by samuel ramey? Samuel Ramey March of Toys Opera Arias Question Holy Night ... recording.artist recording.length recording.artist album.artist 00:04:00 recording.length 00:03:40 KB Target Logical Form (ARGMIN (AND music.recording (JOIN (R artist.track) samuel_ramey)) recording.length) Rank Generate (JOIN (R recording.length) (JOIN recording.artist samuel_ramey)) (AND music.recording (JOIN (R artist.track) samuel_ramey)) (AND music.album (JOIN album.artist samuel_ramey)) Top-Ranked Candidates ... Figure 1: Overview of our rank-and-generate approach. Given a question, we first rank logical form candidates obtained by searching over the KB based on predefined rules. Here, the ground truth logical form is not in the top-ranked candidates as it is not covered by the rules. We solve this problem using another generation step that produces the correct logical form based on top- ranked candidates. The final logical form is executed …

Content truncated to 500 words