Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2704–2713 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2704 Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, Bowen Zhou JD AI Research {ming.tu,guangtao.wang,jing.huang,yun.tang,xiaodong.he,bowen.zhou}@jd.com Abstract Multi-hop reading comprehension (RC) across documents poses new challenge over single- document RC because it requires reasoning over multiple documents to reach the final answer. In this paper, we propose a new model to tackle the multi-hop RC problem. We introduce a heterogeneous graph with different types of nodes and edges, which is named as Heterogeneous Document-Entity (HDE) graph. The advantage of HDE graph is that it contains different granularity lev- els of information including candidates, doc- uments and entities in specific document con- texts. Our proposed model can do reasoning over the HDE graph with nodes representation initialized with co-attention and self-attention based context encoders. We employ Graph Neural Networks (GNN) based message pass- ing algorithms to accumulate evidences on the proposed HDE graph. Evaluated on the blind test set of the Qangaroo WIKIHOP data set, our HDE graph based single model delivers competitive result, and the ensemble model achieves the state-of-the-art performance. 1 Introduction Being able to comprehend a document and out- put correct answer given a query/question about content in the document, often referred as machine reading comprehension (RC) or question answer- ing (QA), is an important and challenging task in natural language processing (NLP). Plenty of data sets have been constructed to facilitate research on this topic, such as SQuAD (Rajpurkar et al., 2016, 2018), NarrativeQA (Koˇcisk`y et al., 2018) and CoQA (Reddy et al., 2018). Many neural models have been proposed to tackle the machine RC/QA problem (Seo et al., 2016; Xiong et al., 2016; Tay et al., 2018), and great success has been achieved, especially after the release of the BERT (Devlin et al., 2018). Query: record label get ready Support doc 1: Mason Durell Betha (born August 27, 1977), better known by stage name Mase (formerly often stylized Ma$e or MA$E), is an American hip hop record- ing artist and minister. He is best known for being signed to Sean “Diddy” Combs’s label Bad Boy Records. . . . Support doc 2: “Get Ready” was the only single re- leased from Mase’s second album, Double Up. It was released on May 25, 1999, produced by Sean “Puffy” Combs, Teddy Riley and Andreao “Fanatic” Heard and featured R&B group, Blackstreet, it contains a sample of “A Night to Remember”, performed by Shalamar. . . . Support doc 3: Bad Boy Entertainment (also known as Bad Boy Records) is an American record label founded in 1993 by Sean Combs. . . . Candidates: bad boy records, record label, rock music, . . . Answer: bad boy records Figure 1: A WIKIHOP example. Words with different colors indicate the evidences across documents. However, current research mainly focuses on machine RC/QA …

Content truncated to 500 words