Learning Physical Common Sense as Knowledge Graph Completion via BERT Data Augmentation and Constrained Tucker Factorization

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3293–3298, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3293 Learning Physical Common Sense as Knowledge Graph Completion via BERT Data Augmentation and Constrained Tucker Factorization Zhenjie Zhao1 Evangelos E. Papalexakis2 Xiaojuan Ma1 1 Department of Computer Science & Engineering, HKUST 2 Department of Computer Science & Engineering, University of California, Riverside zzhaoao@connect.ust.hk, epapalex@cs.ucr.edu, mxj@cse.ust.hk Abstract Physical common sense plays an essential role in the cognition abilities of robots for human- robot interaction. Machine learning methods have shown promising results on physical com- monsense learning in natural language pro- cessing but still suffer from model generaliza- tion. In this paper, we formulate physical com- monsense learning as a knowledge graph com- pletion problem to better use the latent rela- tionships among training samples. Compared with completing general knowledge graphs, completing a physical commonsense knowl- edge graph has three unique characteristics: training data are scarce, not all facts can be mined from existing texts, and the number of relationships is small. To deal with these problems, we first use a pre-training language model BERT to augment training data, and then employ constrained tucker factorization to model complex relationships by constrain- ing types and adding negative relationships. We compare our method with existing state-of- the-art knowledge graph embedding methods and show its superior performance. 1 Introduction Physical common sense means understanding the physical properties of objects and how they can be manipulated (Forbes et al., 2019). Empowering natural language processing (NLP) methods with physical common sense is important when dealing with tasks that are related to the physical world, such as physical commonsense reasoning (Bisk et al., 2020), grounded verb semantics (She and Chai, 2017), and the more general human-robot interaction problem. Generally, there are currently three methods of learning physical common sense: manual annota- tion, text mining, and machine learning. Manual annotation is difficult for human annotators due to inconsistent perceptions and the challenge of enu- merating all physical facts. Mining text data is also challenging because some physical facts are not written in texts explicitly. Machine learning is a promising method to discover new physical facts using existing data. Forbes et al. (2019) formulate physical commonsense learning as three separate machine learning tasks: 1) given an object and a property, predicting whether they follow an object- property (OP) relationship, e.g., an apple is edible; 2) given an object and an affordance, predicting whether they follow an object-affordance (OA) re- lationship, e.g., he drove the car; and 3) given an affordance and a property, predicting whether they follow an affordance-property (AP) relation- ship, e.g., if you can eat something, then it is edi- ble. However, it is difficult for a machine learning model to generalize through the use of the latent relationships among samples. For example, even if we have a training sample an apple is edible, it is hard to say that the trained model can generalize to predict a testing sample an apple is red correctly. …

Content truncated to 500 words