Learning beyond Datasets: Knowledge Graph Augmented Neural Networks for Natural Language Processing

Proceedings of NAACL-HLT 2018, pages 313–322 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing Annervaz K M∗ Indian Institute of Science, Accenture Technology Labs annervaz@iisc.ac.in Somnath Basu Roy Chowdhury∗† IIT Kharagpur brcsomnath@ee.iitkgp.ernet.in Ambedkar Dukkipati Indian Institute of Science ambedkar@iisc.ac.in Abstract Machine Learning has been the quintessen- tial solution for many AI problems, but learn- ing models are heavily dependent on spe- cific training data. Some learning models can be incorporated with prior knowledge using a Bayesian setup, but these learning models do not have the ability to access any orga- nized world knowledge on demand. In this work, we propose to enhance learning models with world knowledge in the form of Knowl- edge Graph (KG) fact triples for Natural Lan- guage Processing (NLP) tasks. Our aim is to develop a deep learning model that can ex- tract relevant prior support facts from knowl- edge graphs depending on the task using atten- tion mechanism. We introduce a convolution- based model for learning representations of knowledge graph entity and relation clusters in order to reduce the attention space. We show that the proposed method is highly scalable to the amount of prior information that has to be processed and can be applied to any generic NLP task. Using this method we show sig- nificant improvement in performance for text classification with 20Newsgroups (News20) & DBPedia datasets, and natural language infer- ence with Stanford Natural Language Infer- ence (SNLI) dataset. We also demonstrate that a deep learning model can be trained with sub- stantially less amount of labeled training data, when it has access to organized world knowl- edge in the form of a knowledge base. 1 Introduction Today, machine learning is centered around al- gorithms that can be trained on available task- specific labeled and unlabeled training samples. Although learning paradigms like Transfer Learn- ing (Pan and Yang, 2010) attempt to incorporate ∗equal contribution †Main work done during internship at Accenture Technol- ogy Labs knowledge from one task into another, these tech- niques are limited in scalability and are specific to the task at hand. On the other hand, humans have the intrinsic ability to elicit required past knowl- edge from the world on demand and infuse it with newly learned concepts to solve problems. The question that we address in this paper is the following: Is it possible to develop learning mod- els that can be trained in a way that it is able to infuse a general body of world knowledge for pre- diction apart from learning based on training data? World- Knowledge Base X + Xw X ′ Y f Figure 1: The Basic Idea: X is the feature input and Y is the prediction. The relevant world knowledge for the task Xw, is retrieved and augmented with the feature input before making the final prediction By world knowledge, we mean structured gen- eral purpose knowledge that need not be domain specific. …

Content truncated to 500 words