TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4149–4158 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4149 TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph Jiaxin Shi1,2 , Shulin Cao1 , Lei Hou1∗, Juanzi Li1 and Hanwang Zhang3 1Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing 100084, China 2Cloud BU, Huawei Technologies, 3Nanyang Technological University shijx12@gmail.com {caosl19@mails., houlei@, lijuanzi@}tsinghua.edu.cn hanwangzhang@ntu.edu.sg Abstract Multi-hop Question Answering (QA) is a chal- lenging task because it requires precise rea- soning with entity relations at every step to- wards the answer. The relations can be rep- resented in terms of labels in knowledge graph (e.g., spouse) or text in text corpus (e.g., they have been married for 26 years). Existing models usually infer the answer by predict- ing the sequential relation path or aggregat- ing the hidden graph features. The former is hard to optimize, and the latter lacks inter- pretability. In this paper, we propose Trans- ferNet, an effective and transparent model for multi-hop QA, which supports both label and text relations in a unified framework. Trans- ferNet jumps across entities at multiple steps. At each step, it attends to different parts of the question, computes activated scores for relations, and then transfer the previous en- tity scores along activated relations in a dif- ferentiable way. We carry out extensive ex- periments on three datasets and demonstrate that TransferNet surpasses the state-of-the-art models by a large margin. In particular, on MetaQA, it achieves 100% accuracy in 2-hop and 3-hop questions. By qualitative analysis, we show that TransferNet has transparent and interpretable intermediate results. 1 Introduction Question answering (QA) plays a central role in artificial intelligence. It requires machines to un- derstand the free-form questions and infer the an- swers by analyzing information from a large cor- pus (Rajpurkar et al., 2016; Joshi et al., 2017; Chen et al., 2017) or structured knowledge base (Bordes et al., 2015; Yih et al., 2015; Jiang et al., 2019). Along with the fast development of deep learn- ing, especially the pretraining technology (Devlin et al., 2018; Lan et al., 2019), state-of-the-art mod- els have been shown comparative with human per- ∗Corresponding author. Bill Gates Melinda Gates Bill & Melinda Gates Foundation Microsoft Corporation <sub> and <obj> have been married for 26 years. In 2000, Melinda Gates co-founded the <obj> with her husband <sub>. In 2000, <sub> co- founded the <obj> with her husband Bill Gates. During his career at <sub>, <obj> held the positions of chairman, chief executive officer (CEO), president and chief software architect. Bill Gates Melinda Gates Bill & Melinda Gates Foundation Microsoft Corporation Spouse Founder CEO Founder Label Form Text Form Bill & Melinda Gates Foundation Question: Answer: What the wife of Bill Gates found? organization did Relation Graph Figure 1: Answering a multi-hop question over the relation graph. The relations are constrained predi- cates in the label form (i.e., knowledge graph) while free texts in the text form. The reasoning process …

Content truncated to 500 words