Is Graph Structure Necessary for Multi-hop Question Answering?

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7187–7192, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7187 Is Graph Structure Necessary for Multi-hop Question Answering? Nan Shao†, Yiming Cui‡†, Ting Liu‡, Shijin Wang†§, Guoping Hu† †State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China ‡Research Center for Social Computing and Information Retrieval (SCIR), Harbin Institute of Technology, Harbin, China §iFLYTEK AI Research (Hebei), Langfang, China †§{nanshao,ymcui,sjwang3,gphu}@iflytek.com ‡{ymcui,tliu}@ir.hit.edu.cn Abstract Recently, attempting to model texts as graph structure and introducing graph neural net- works to deal with it has become a trend in many NLP research areas. In this paper, we investigate whether the graph structure is nec- essary for multi-hop question answering. Our analysis is centered on HotpotQA. We con- struct a strong baseline model to establish that, with the proper use of pre-trained mod- els, graph structure may not be necessary for multi-hop question answering. We point out that both graph structure and adjacency matrix are task-related prior knowledge, and graph- attention can be considered as a special case of self-attention. Experiments and visualized analysis demonstrate that graph-attention or the entire graph structure can be replaced by self-attention or Transformers. 1 Introduction Different from single-hop question answering, where the answer can be derived from a single sen- tence in a single paragraph, more and more studies focus on multi-hop question answering across mul- tiple documents or paragraphs (Welbl et al., 2018; Talmor and Berant, 2018; Yang et al., 2018). To solve this problem, the majority of existing studies constructed a graph structure according to co-occurrence relations of entities that scattered across multiple sentences or paragraphs. Dhin- gra et al. (2018) and Song et al. (2018) designed a DAG-styled recurrent layer to model the rela- tions between entities. De Cao et al. (2019) first used GCN (Kipf and Welling, 2017) to tackle en- tity graph. Qiu et al. (2019) proposed a dynamic entity graph for span-based multi-hop QA. Tu et al. (2019b) extended the entity graph to a heteroge- neous graph by introducing document nodes and query nodes. Previous works argue that a fancy graph structure is a vital part of their models and demonstrate that by ablation experiments. However, in experiments, we find when we use the pre-trained models in the fine-tuning approach, removing the entire graph structure may not hurt the final results. Therefore, in this paper, we aimed to answer the following question: How much does graph structure con- tribute to multi-hop question answering? To answer the question above, we choose the widely used multi-hop question answering bench- mark, HotpotQA (Yang et al., 2018), as our testbed. We reimplement a graph-based model, Dynami- cally Fused Graph Network (Qiu et al., 2019), as our baseline model. The remainder of this paper is organized as follows. • In Section 2, we first describe our baseline model. Then, we show that the graph structure can play an important role only when the pre-trained mod- els are used in a feature-based manner. While the pre-trained models …

Content truncated to 500 words