Machine Reading Comprehension Using Structural Knowledge Graph-aware Network

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5896–5901, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5896 Machine Reading Comprehension Using Structural Knowledge Graph-aware Network Delai Qiu1, Yuanzhe Zhang2, Xinwei Feng4, Xiangwen Liao1, Wenbin Jiang4, Yajuan Lyu4, Kang Liu2,3, Jun Zhao2,3 1 College of Mathematics and Computer Science, Fuzhou University, Fuzhou, China 2 Institute of Automation, Chinese Academy of Sciences, Beijing, China 3 University of Chinese Academy of Sciences, Beijing, China 4 Baidu Inc., Beijing, China noneqdl@gmail.com, {yzzhang,kliu,jzhao}@nlpr.ia.ac.cn lxw@fzu.edu.cn, {fengxinwei,jiangwenbin,lvyajuan}@baidu.com Abstract Leveraging external knowledge is an emerg- ing trend in machine comprehension task. Pre- vious work usually utilizes knowledge graph- s such as ConceptNet as external knowledge, and extracts triples from them to enhance the initial representation of the machine compre- hension context. However, such method can- not capture the structural information in the knowledge graph. To this end, we propose a Structural Knowledge Graph-aware Network (SKG) model, constructing sub-graphs for en- tities in the machine comprehension context. Our method dynamically updates the represen- tation of the knowledge according to the struc- tural information of the constructed sub-graph. Experiments show that SKG achieves state-of- the-art performance on the ReCoRD dataset. 1 Introduction Machine reading comprehension (MRC) is an im- portant subtask in natural language processing, which requires a system to read a given passage and answer questions about it. The ability of utilizing external knowledge is of great signifi- cance in an MRC system (Rajpurkar et al., 2016; Trischler et al., 2017). Latest large-scale datasets, e.g. ReCoRD (Zhang et al., 2018) specify that ex- ternal knowledge is required to answer questions. Many previous studies have introduced exter- nal knowledge in machine comprehension (Weis- senborn, 2017; Mihaylov and Frank, 2018; Bauer et al., 2018). They often acquire external knowl- edge from structural knowledge graphs, such as ConceptNet (Speer et al., 2017) and Free- base (Tanon et al., 2016), in which knowledge is organized by triples like “(shortage, related to, lack)” and “(need, related to, lack)”. However, most of them fail to make full use of the struc- tural information in the knowledge graph, and use sequence modeling methods like recurrent neural networks (RNNs) to generate the representation of knowledge, rather than based on graph structure. In this paper, we present a Structural Knowl- edge Graph-aware Network (SKG) model to lever- age the structural information of external knowl- edge. The advantages of our proposed method are two folds: a) the constructed sub-graphs con- tain both MRC context and external knowledge nodes, reserving the knowledge structure; b) the graph neural network is capable of updating the nodes dynamically according to the structure of sub-graphs, instead of using external knowledge as pre-trained representations. Concretely, we first construct sub-graphs from external knowl- edge such as ConceptNet based on the context, and initialize the representation of nodes via knowl- edge graph embedding method (Yang et al., 2015). Then we employ graph attention networks to dy- namically update …

Content truncated to 500 words