Large-Scale Relation Learning for Question Answering over Knowledge Bases with Pre-trained Language Models

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3653–3660 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3653 Large-Scale Relation Learning for Question Answering over Knowledge Bases with Pre-trained Language Models Yuanmeng Yan1, Rumei Li2, Sirui Wang2, Hongzhi Zhang2, Daoguang Zan3 Fuzheng Zhang2, Wei Wu2, Weiran Xu1∗ 1Beijing University of Posts and Telecommunications, Beijing, China 2Meituan Inc., Beijing, China 3University of Chinese Academy of Sciences, Beijing, China {yanyuanmeng,xuweiran}@bupt.edu.cn {lirumei,wangsirui,zhanghongzhi}@meituan.com Abstract The key challenge of question answering over knowledge bases (KBQA) is the inconsistency between the natural language questions and the reasoning paths in the knowledge base (KB). Recent graph-based KBQA methods are good at grasping the topological structure of the graph but often ignore the textual informa- tion carried by the nodes and edges. Mean- while, pre-trained language models learn mas- sive open-world knowledge from the large cor- pus, but it is in the natural language form and not structured. To bridge the gap between the natural language and the structured KB, we propose three relation learning tasks for BERT- based KBQA, including relation extraction, re- lation matching, and relation reasoning. By relation-augmented training, the model learns to align the natural language expressions to the relations in the KB as well as reason over the missing connections in the KB. Experiments on WebQSP show that our method consistently outperforms other baselines, especially when the KB is incomplete. 1 Introduction Question Answering over Knowledge Base (KBQA) aims to find the answers to a natural lan- guage question given the structured knowledge base (KB) and is widely used in modern ques- tion answering and information retrieval systems. Traditional retrieval-based KBQA approaches typi- cally build it as a pipeline system, including name entity recognization, entity linking, subgraph re- trieval, and entity scoring. In recent years, with the help of deep representation learning, such ap- proaches have achieved remarkable performance (Dong et al., 2015; Miller et al., 2016; Xu et al., 2016; Sun et al., 2018, 2019; Saxena et al., 2020; He et al., 2021). ∗Work done during internship at Meituan Inc. Weiran Xu is the corresponding author. Question: what is monta ellis career high points ? Monta Ellis CVT1 NBA Most Improved Player Award CVT2 Lanier High School Correct Answer: NBA Most Improved Player Award Model Prediction: Lanier High School a) Shallow Matching: The model fails to interpret “career high points” as “honor” or “award”, but matched “high school” according to the surface similarity (the word “high”). Score: 0.01 Score: 0.59 × √ Question: what is the name of king george vi wife ? Correct Answer: Queen Elizabeth The Queen Mother Model Prediction: 1923-04-26 √× George VI CVT b) Incomplete KB: The model matches “wife” to “spouse” and “marriage”. However, some connections are missing from George VI to Queen Elizabeth. On the other hand, the model also fails to reason over the path “George VI – people.person.children – Princess Margaret – people.person.parents – Queen Elizabeth The Queen Mother” 1923-04-26 Queen Elizabeth The Queen Mother Princess Margaret Score: 0.05 …

Content truncated to 500 words