KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2829–2839, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2829 KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning Bill Yuchen Lin† Xinyue Chen‡ Jamin Chen† Xiang Ren† {yuchen.lin,jaminche,xiangren}@usc.edu, kiwisher@sjtu.edu.cn †Computer Science Department, University of Southern California ‡Computer Science Department, Shanghai Jiao Tong University Abstract Commonsense reasoning aims to empower machines with the human ability to make presumptions about ordinary situations in our daily life. In this paper, we propose a textual inference framework for answer- ing commonsense questions, which effec- tively utilizes external, structured common- sense knowledge graphs to perform explain- able inferences. The framework first grounds a question-answer pair from the semantic space to the knowledge-based symbolic space as a schema graph, a related sub-graph of exter- nal knowledge graphs. It represents schema graphs with a novel knowledge-aware graph network module named KAGNET, and finally scores answers with graph representations. Our model is based on graph convolutional networks and LSTMs, with a hierarchical path-based attention mechanism. The interme- diate attention scores make it transparent and interpretable, which thus produce trustworthy inferences. Using ConceptNet as the only external resource for BERT-based models, we achieved state-of-the-art performance on the CommonsenseQA, a large-scale dataset for commonsense reasoning. We open-source our code1 to the community for future research in knowledge-aware commonsense reasoning. 1 Introduction Human beings are rational and a major component of rationality is the ability to reason. Reasoning is the process of combining facts and beliefs to make new decisions (Johnson-Laird, 1980), as well as the ability to manipulate knowledge to draw in- ferences (Hudson and Manning, 2018). Common- sense reasoning utilizes the basic knowledge that reflects our natural understanding of the world and human behaviors, which is common to all humans. 1https://github.com/INK-USC/KagNet Where do adults use glue sticks? A: classroom B: office C: desk drawer Semantic Space Symbolic Space glue_stick adult work CapableOf AtLocation AtLocation office use HasSubevent CapableOf ReceiveAction Grounding Knowledge-Aware Commonsense Inference Schema Graph Where do adults use glue sticks? A: classroom B: office C: desk drawer glue_stick adult work CapableOf AtLocation AtLocation office use HasSubevent CapableOf ReceiveAction Grounding Knowledge-Aware Commonsense Inference Schema Graph Figure 1: An example of using external commonsense knowledge (symbolic space) for inference in natural language commonsense questions (semantic space). Empowering machines with the ability to per- form commonsense reasoning has been seen as the bottleneck of artificial general intelligence (Davis and Marcus, 2015). Recently, there have been a few emerging large-scale datasets for testing ma- chine commonsense with various focuses (Zellers et al., 2018; Sap et al., 2019b; Zellers et al., 2019). In a typical dataset, CommonsenseQA (Talmor et al., 2019), given a question like “Where do adults use glue sticks?”, with the answer choices being {classroom(), office (), desk drawer ()}, a commonsense reasoner is expected to differenti- ate the correct choice from other “distractive” can- didates. False choices are usually highly related to the question context, …

Content truncated to 500 words