Cognitive Graph for Multi-Hop Reading Comprehension at Scale

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2694–2703 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2694 Cognitive Graph for Multi-Hop Reading Comprehension at Scale Ming Ding†, Chang Zhou‡, Qibin Chen†, Hongxia Yang‡, Jie Tang† †Department of Computer Science and Technology, Tsinghua University ‡DAMO Academy, Alibaba Group {dm18,chen-qb15}@mails.tsinghua.edu.cn {ericzhou.zc,yang.yhx}@alibaba-inc.com jietang@tsinghua.edu.cn Abstract We propose a new CogQA framework for multi-hop question answering in web-scale documents. Founded on the dual process the- ory in cognitive science, the framework grad- ually builds a cognitive graph in an iterative process by coordinating an implicit extrac- tion module (System 1) and an explicit rea- soning module (System 2). While giving ac- curate answers, our framework further pro- vides explainable reasoning paths. Specifi- cally, our implementation1 based on BERT and graph neural network (GNN) efficiently handles millions of documents for multi-hop reasoning questions in the HotpotQA fullwiki dataset, achieving a winning joint F1 score of 34.9 on the leaderboard, compared to 23.6 of the best competitor.2 1 Introduction Deep learning models have made significant strides in machine reading comprehension and even outperformed human on single paragraph question answering (QA) benchmarks including SQuAD (Wang et al., 2018b; Devlin et al., 2018; Rajpurkar et al., 2016). However, to cross the chasm of reading comprehension ability between machine and human, three main challenges lie ahead: 1) Reasoning ability. As revealed by ad- versarial tests (Jia and Liang, 2017), models for single paragraph QA tend to seek answers in sen- tences matched by the question, which does not involve complex reasoning. Therefore, multi-hop QA becomes the next frontier to conquer (Yang et al., 2018). 2) Explainability. Explicit rea- soning paths, which enable verification of logi- cal rigor, are vital for the reliability of QA sys- tems. HotpotQA (Yang et al., 2018) requires models to provide supporting sentences, which 1Codes: https://github.com/THUDM/CogQA 2https://hotpotqa.github.io, March 4, 2019 ������������ ����������� ����������� �������������������� ���������������� ������������������ ��������������������������������������������������������������� ����������������������������������������������� ������������������� ����������� ��������������� ���������� ��������������������������������� ������������������������������� ��������������������� ��������� ��������������������� ��������������������� ������������ �������������� ������������������������ ��������������������������� ���� ������������������������� ������������ ������������������ ��������������� ���������������� ���������������� �������� ����� �������� �������� ���� ����� ����� �������� ������ ����� Figure 1: An example of cognitive graph for multi-hop QA. Each hop node corresponds to an entity (e.g., “Los Angeles”) followed by its introductory paragraph. The circles mean ans nodes, answer candidates to the ques- tion. Cognitive graph mimics human reasoning pro- cess. Edges are built when calling an entity to “mind”. The solid black edges are the correct reasoning path. means unordered and sentence-level explainabil- ity, yet humans can interpret answers with step by step solutions, indicating an ordered and entity- level explainability. 3) Scalability. For any prac- tically useful QA system, scalability is indis- pensable. Existing QA systems based on ma- chine comprehension generally follow retrieval- extraction framework in DrQA (Chen et al., 2017), reducing the scope of sources to a few paragraphs by pre-retrieval. This framework is a simple com- promise between single paragraph QA and scal- able information retrieval, compared to human’s …

Content truncated to 500 words