Relation-aware Bidirectional Path Reasoning for Commonsense Question Answering

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 445–453 November 10–11, 2021. ©2021 Association for Computational Linguistics 445 Relation-aware Bidirectional Path Reasoning for Commonsense Question Answering Junxing Wang, Xinyi Li, Zhen Tan, Xiang Zhao, Weidong Xiao National University of Defense Technology, China {junxingwang,xinyili,tanzhen08a,xiangzhao,wdxiao}@nudt.edu.cn Abstract Commonsense Question Answering is an im- portant natural language processing (NLP) task that aims to predict the correct answer to a question through commonsense reasoning. Previous studies utilize pre-trained models on large-scale corpora such as BERT, or perform reasoning on knowledge graphs. However, these methods do not explicitly model the rela- tions that connect entities, which are informa- tional and can be used to enhance reasoning. To address this issue, we propose a relation- aware reasoning method. Our method uses a relation-aware graph neural network to cap- ture the rich contextual information from both entities and relations. Compared with meth- ods that use fixed relation embeddings from pre-trained models, our model dynamically updates relations with contextual information from a multi-source subgraph, built from mul- tiple external knowledge sources. The en- hanced representations of relations are then fed to a bidirectional reasoning module. bidirectional attention mechanism is appliedA between the question sequence and the paths that connect entities, which provides us with transparent interpretability. Experimental re- sults on the CommonsenseQA dataset illus- trate that our method results in significant im- provements over the baselines while also pro- viding clear reasoning paths. 1 Introduction Commonsense Question Answering (CQA) is a task that requires machines to not only understand the question, but also infer through external knowl- edge. For example, to answer the question in Fig- ure 1 (“What do people typically do while playing guitar?”), it is necessary to discover the connec- tion between the evidence “playing guitar” and “singing”. Although seemingly trivial for humans, knowledge like this is difficult to learn for natural language understanding (NLU) models. Q:What do people typically do while playing guitar? A. cry B. hear sounds C. singing D. anthritis E. making music Figure 1: An example of CommonsenseQA. Pre-trained language models (PTLMs) with dis- tributed representations, such as BERT, RoBERTa, XLNet and ELECTRA(Devlin et al., 2019; Liu et al., 2019; Yang et al., 2019; Clark et al., 2020), have performed well on many Natural Language Processing (NLP) tasks. However, when answer- ing commonsense questions, these models fail to explicitly recover the reasoning process. Specifi- cally, for the question in Figure 1, humans figure out the correct answer by inferring from the rela- tion between “playing guitar” and “singing”, while PTLMs only provide a predicted answer and can not explain the inference process. They are thus lacking in transparency and interpretability. Appar- ently, there is still a wide gap between the PTLMs and human level performance in CQA tasks. Since merely relying on pre-trained large lan- guage models can not utilize commonsense knowl- edge for reasoning, a straightforward solution is to model the paths(Lin et al., 2019) in the knowl- edge graph “that connect pairs of concepts”. Kag- Net(Lin …

Content truncated to 500 words