ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 401–410 Barcelona, Spain (Online), December 12, 2020. 401 ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation Qian Zhao and Siyu Tao and Jie Zhou and Linlin Wang ∗and Xin Lin ∗and Liang He School of Computer Science and Technology, East China Normal University, Shanghai 200062, China {im0qianqian, sytao, jzhou}@ica.stc.sh.cn {llwang, xlin, lhe}@cs.ecnu.edu.cn Abstract This paper describes our system for SemEval-2020 Task 4: Commonsense Validation and Expla- nation (Wang et al., 2020). We propose a novel Knowledge-enhanced Graph Attention Network (KEGAT) architecture for this task, leveraging heterogeneous knowledge from both the structured knowledge base (i.e. ConceptNet) and unstructured text to better improve the ability of a machine in commonsense understanding. This model has a powerful commonsense inference capabil- ity via utilizing suitable commonsense incorporation methods and upgraded data augmentation techniques. Besides, an internal sharing mechanism is cooperated to prohibit our model from insufficient and excessive reasoning for commonsense. As a result, this model performs quite well in both validation and explanation. For instance, it achieves state-of-the-art accuracy in the subtask called Commonsense Explanation (Multi-Choice). We officially name the system as ECNU- SenseMaker. Code is publicly available at https://github.com/ECNU-ICA/ECNU-SenseMaker. 1 Introduction Commonsense reasoning is the process of making decisions by combining facts and beliefs with some basic knowledge appeared in daily life, which is fundamental to many Natural Language Understanding (NLU) tasks (Johnson-Laird, 1980). However, most existing models are quite weak in terms of commonsense acquisition and understanding compared with humans since commonsense is often expressed implicitly and constantly evolving (Wang et al., 2019a). Thus, the commonsense problem is considered to be an important bottleneck of modern NLU systems (Davis and Marcus, 2015). Many attempts have been made to empower machines with human abilities in commonsense under- standing, and several benchmarks have emerged to verify the commonsense reasoning capability, such as SQUABU (Davis, 2016) and CommonsenseQA (Talmor et al., 2019; Lin et al., 2019). However, these benchmarks estimate commonsense indirectly and do not include corresponding explanations. Therefore, the Commonsense Validation and Explanation (ComVE) benchmark is proposed which directly asks the real reasons behind decision making (Wang et al., 2020). It consists of three subtasks, and we propose an entire system (ECNU-SenseMaker) to solve the first two subtasks, namely, Commonsense Validation and Commonsense Explanation (Multi-Choice). In fact, the two different subtasks correlate to each other. Subtask A (Validation) requires the model to select which one is invalid between two given statements. While Subtask B (Explanation) aims to identify one real reason from two other confusing options to explain why the statement is invalid. Compared with the previous benchmarks, these subtasks are non-trivial because machines does not have the ability of children and adults in accumulating commonsense knowledge in daily life. Since most questions in such subtasks require knowledge beyond the mentioned facts in text, even trained over millions of sentences, there still exist a great gap behind human performance in commonsense reasoning, not to mention explanation capability …

Content truncated to 500 words