Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 132–141, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, David Belanger, Andrew McCallum College of Information and Computer Sciences University of Massachusetts, Amherst {rajarshi, arvind, belanger, mccallum}@cs.umass.edu Abstract Our goal is to combine the rich multi- step inference of symbolic logical rea- soning with the generalization capabili- ties of neural networks. We are par- ticularly interested in complex reasoning about entities and relations in text and large-scale knowledge bases (KBs). Nee- lakantan et al. (2015) use RNNs to obtain dense representations of multi-hop paths in KBs; however for multiple reasons, the approach lacks accuracy and practi- cality. This paper proposes three signifi- cant modeling advances: (1) we learn to jointly reason about relations, entities, and entity-types; (2) we use neural attention modeling to incorporate multiple paths; (3) we learn to share strength in a sin- gle RNN that represents logical compo- sition across all relations. On a large- scale Freebase+ClueWeb prediction task, we achieve 25% error reduction, and a 53% error reduction on sparse relations. On chains of reasoning in WordNet we re- duce error in mean quantile by 84% versus the previous state of the art.1. 1 Introduction There is a rising interest in extending neural net- works to perform more complex reasoning, for- merly addressed only by symbolic and logical rea- soning systems. So far this work has mostly fo- cused on small or synthetic data (Grefenstette, 2013; Bowman et al., 2014; Rockt¨aschel and Riedel, 2016). Our interest is primarily in reason- ing about large knowledge bases (KBs) with di- verse semantics, populated from text. One method 1The code and data are available at https://rajarshd.github.io/ChainsofReasoning/ i. place.birthpa, bq Ð ‘was born in’pa, xq^ ‘commonly known as’px, bq ii. location.containspa, bq Ð nationality´1pa, xq ^ place.birthpx, bq iii. book.characterspa, bq Бaka’pa, xq^ (theater.character.plays)´1px, bq iv. cause.deathpa, bq Бcontracted’pa, bq Table 1: Several highly probable clauses learnt by our model. The textual relations are shown in quotes and italicized. Our model has the ability to combine textual and schema relations. r´1 is the inverse relation r, i.e. rpa, bq ô r´1pb, aq. for populating a KB from text (and for repre- senting diverse semantics in the KB) is Universal Schema (Riedel et al., 2013; Verga et al., 2016), which learns vector embeddings of relation types - the union of all input relation types, both from the schemas of multiple structured KBs, as well as expressions of relations in natural language text. An important reason to populate a KB is to support not only look-up-style question answer- ing, but reasoning on its entities and relations in order to make inferences not directly stored in the KB. KBs are often highly incomplete (Min et al., 2013), and reasoning can fill in these missing facts. The “matrix completion” mecha- nism that underlies the common implementation …

Content truncated to 500 words