Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4474–4484, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4474 Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning Yichen Jiang and Mohit Bansal UNC Chapel Hill {yichenj, mbansal}@cs.unc.edu Abstract Multi-hop QA requires a model to connect multiple pieces of evidence scattered in a long context to answer the question. The recently proposed HotpotQA (Yang et al., 2018) dataset is comprised of questions em- bodying four different multi-hop reasoning paradigms (two bridge entity setups, check- ing multiple properties, and comparing two entities), making it challenging for a single neural network to handle all four. In this work, we present an interpretable, controller- based Self-Assembling Neural Modular Net- work (Hu et al., 2017, 2018) for multi-hop rea- soning, where we design four novel modules (Find, Relocate, Compare, NoOp) to perform unique types of language reasoning. Based on a question, our layout controller RNN dy- namically infers a series of reasoning modules to construct the entire network. Empirically, we show that our dynamic, multi-hop modu- lar network achieves significant improvements over the static, single-hop baseline (on both regular and adversarial evaluation). We further demonstrate the interpretability of our model via three analyses. First, the controller can softly decompose the multi-hop question into multiple single-hop sub-questions to promote compositional reasoning behavior of the main network. Second, the controller can predict layouts that conform to the layouts designed by human experts. Finally, the intermediate module can infer the entity that connects two distantly-located supporting facts by address- ing the sub-question from the controller.1 1 Introduction The task of multi-hop question answering (QA) requires the model to answer a natural language question by finding multiple relevant information 1Our code is publicly available at: https://github. com/jiangycTarheel/NMN-MultiHopQA Figure 1: Two HotpotQA examples and the modular network layout predicted by the controller. pieces scattered in a given natural language con- text. It has attracted more attention recently and multiple datasets have been proposed, including the recent HotpotQA (Yang et al., 2018) that is comprised of questions embodying four differ- ent multi-hop reasoning paradigms: inferring the bridge entity to complete the 2nd-hop question (first question in Fig. 1), inferring the answer through a bridge entity, checking multiple proper- ties to select the answer, and comparing two en- tities (second question in Fig. 1). For the first question, it is necessary to first find the person “who portrayed Corliss Archer in the film Kiss and Tell”, and then find out the “government position” she held. For the second question, one may arrive at the answer by finding the country where Scott and Ed are from, and then comparing the two na- tionalities to conclude whether they are the same. Multi-hop QA is more challenging than single- hop QA for two main reasons. First, the tech- niques used for single-hop QA are not sufficient to answer a multi-hop question. In single-hop QA tasks like SQuAD (Rajpurkar et …

Content truncated to 500 words