Deep Inductive Logic Reasoning for Multi-Hop Reading Comprehension

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4999 - 5009 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Deep Inductive Logic Reasoning for Multi-Hop Reading Comprehension Wenya Wang and Sinno Jialin Pan Nanyang Technological University, Singapore {wangwy, sinnopan}@ntu.edu.sg Abstract Multi-hop reading comprehension requires an ability to reason across multiple documents. On the one hand, deep learning approaches only implicitly encode query-related information into distributed embeddings which fail to un- cover the discrete relational reasoning process to infer the correct answer. On the other hand, logic-based approaches provide interpretable rules to infer the target answer, but mostly work on structured data where entities and relations are well-defined. In this paper, we propose a deep-learning based inductive logic reason- ing method that firstly extracts query-related (candidate-related) information, and then con- ducts logic reasoning among the filtered infor- mation by inducing feasible rules that entail the target relation. The reasoning process is ac- complished via attentive memories with novel differentiable logic operators. To demonstrate the effectiveness of our model, we evaluate it on two reading comprehension datasets, namely WikiHop and MedHop. 1 Introduction Reasoning has been extensively studied in the struc- tured domain, e.g., knowledge base completion which infers missing facts given background enti- ties and relations. However, when the background knowledge is expressed in natural languages, as shown in the multi-hop reading comprehension problem with triplet-form questions (Welbl et al., 2018), it becomes difficult to conduct complex rea- soning because the entities and relations are not explicitly labeled in the documents. For exam- ple, consider the question “country(Moonhole, ?)”, given the following documents: “Moonhole is a private community on the island of Bequia. Moonhole was founded by Thomas and Gladys Johnston in the 1960s.” “Gladys Johnston was born in United States.” “Bequia is an island and is part of the country of Saint Vincent and the Grenadines” In this example, the underlined entities are used to infer the correct answer, i.e., “country(Moonhole, Saint Vincent and the Grenadines)”, but are not explicitly annotated for relational reasoning. Deep neural networks (DNNs) for multi-hop reading comprehension (RC) can be summarized into following three categories. 1) Memory-based models (Zhong et al., 2019; Wang et al., 2018; Zhuang and Wang, 2019) that produce query- aware context representations. 2) Graph-based ap- proaches (Song et al., 2018; De Cao et al., 2019) that use graph neural networks to propagate infor- mation based on pre-constructed entity (context) graphs. 3) Neural Module networks (Andreas et al., 2016) that decompose the question into a series of action modules (Jiang and Bansal, 2019; Gupta et al., 2020). However, DNNs only implicitly en- code relevant contexts and fail to explicitly uncover the underlying relational compositions for com- plex inference. For instance, in the above example, DNNs may encode Bequia and Gladys Johnson into 1-hop features, given the fact that both entities co-occur with the query Moonhole. As a result, the model may predict United States by linking it with Gladys Johnson instead of the …

Content truncated to 500 words