CORN: Co-Reasoning Network for Commonsense Question Answering

Proceedings of the 29th International Conference on Computational Linguistics, pages 1677–1686 October 12–17, 2022. 1677 CORN: Co-Reasoning Network for Commonsense Question Answering Xin Guan, Biwei Cao, Qingqing Gao, Zheng Yin, Bo Liu, Jiuxin Cao∗ Southeast University {xin_guan,caobiwei,qingqing_gao,z.yin,bliu,jx.cao}@seu.edu.cn Abstract Commonsense question answering (QA) re- quires machines to utilize the QA content and external commonsense knowledge graph (KG) for reasoning when answering questions. Ex- isting work uses two independent modules to model the QA contextual text representa- tion and relationships between QA entities in KG, which prevents information sharing be- tween modules for co-reasoning. In this pa- per, we propose a novel model, Co-Reasoning Network (CORN), which adopts a bidirectional multi-level connection structure based on Co- Attention Transformer. The structure builds bridges to connect each layer of the text en- coder and graph encoder, which can introduce the QA entity relationship from KG to the text encoder and bring contextual text infor- mation to the graph encoder, so that these fea- tures can be deeply interactively fused to form comprehensive text and graph node represen- tations. Meanwhile, we propose a QA-aware node based KG subgraph construction method. The QA-aware nodes aggregate the question entity nodes and the answer entity nodes, and further guide the expansion and construction process of the subgraph to enhance the con- nectivity and reduce the introduction of noise. We evaluate our model on QA benchmarks in the CommonsenseQA and OpenBookQA datasets, and CORN achieves state-of-the-art performance. 1 Introduction Commonsense Question answering (QA) research requires the machine to have a human thought pat- tern, which is capable of comprehending text con- tent and combining commonsense knowledge to reason and arrive at the correct answer. Despite the success of large pre-trained language models (PLMs) (Devlin et al., 2019; Liu et al., 2019) on various NLP tasks, there is still a large gap be- tween PLMs and humans on commonsense QA ∗Corresponding author. GNN Layer Transformer Encoder Subgraph QA Context Transformer Encoder Transformer Encoder … GNN Layer GNN Layer … PLM GNN ⨁ Score Output Text Encoder Graph Encoder No Connection No Connection No Connection Figure 1: The network architecture of current com- monsense QA research. Text encoder for encoding the QA content, graph encoder for reasoning on the graph. There is no connection between stacked text layers and stacked GNN layers. tasks. Therefore, researchers try to introduce exter- nal knowledge, such as Freebase (Bollacker et al., 2008) and ConceptNet (Speer et al., 2017), which are large knowledge graphs (KGs) that entities link by various relationships. There has already been a significant amount of work that combines PLMs and KGs for reasoning (Lin et al., 2019; Wang et al., 2020; Feng et al., 2020; Yasunaga et al., 2021). As illustrated in Fig- ure 1, these works are mainly composed of two modules: (1) capturing text features on QA with a text encoder (such as PLM). (2) extracting sub- graph from KG and reasoning on it with a graph encoder, such as the GNN-based model (Kipf and Welling, 2017; Schlichtkrull et al., …

Content truncated to 500 words