Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources

Proceedings of the 28th International Conference on Computational Linguistics, pages 2583–2594 Barcelona, Spain (Online), December 8-13, 2020 2583 Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources Qianglong Chen1, Feng Ji2, Haiqing Chen2, Yin Zhang1∗ 1College of Computer Science and Technology, Zhejiang University, China 2DAMO Academy, Alibaba Group, China {chenqianglong,zhangyin98}@zju.edu.cn {zhongxiu.jf, haiqing.chenhq}@alibaba-inc.com Abstract In order to facilitate natural language understanding, the key is to engage commonsense or back- ground knowledge. However, how to engage commonsense effectively in question answering systems is still under exploration in both research academia and industry. In this paper, we pro- pose a novel question-answering method by integrating multiple knowledge sources, i.e. Con- ceptNet, Wikipedia, and the Cambridge Dictionary, to boost the performance. More concretely, we first introduce a novel graph-based iterative knowledge retrieval module, which iteratively retrieves concepts and entities related to the given question and its choices from multiple knowl- edge sources. Afterward, we use a pre-trained language model to encode the question, retrieved knowledge and choices, and propose an answer choice-aware attention mechanism to fuse all hidden representations of the previous modules. Finally, the linear classifier for specific tasks is used to predict the answer. Experimental results on the CommonsenseQA dataset show that our method significantly outperforms other competitive methods and achieves the new state-of- the-art. In addition, further ablation studies demonstrate the effectiveness of our graph-based iterative knowledge retrieval module and the answer choice-aware attention module in retrieving and synthesizing background knowledge from multiple knowledge sources. 1 Introduction In the past decade, most previous work on question answering systems can be divided into two cat- egories, namely, answering questions within a given context (Rajpurkar et al., 2016; Mihaylov et al., 2018) or without any context (Talmor et al., 2018; Yang et al., 2018; Sap et al., 2019b). Although some methods (Lan et al., 2020; Zhang et al., 2020) have been reported to exceed human performance on a few metrics, those methods are seldom involved with external knowledge, such as commonsense or back- ground knowledge, which is necessary for better understanding natural language questions, especially in some settings that do not mention the background knowledge. To effectively leverage external human-made knowledge graphs, previous methods (Chen et al., 2017; Min et al., 2019; Lin et al., 2019a; Lv et al., 2019; Ye et al., 2019; Shen et al., 2019) adopt a two-stage framework, in which the first stage is to find knowledge facts related to a given question from a wide range of knowledge sources, and then the second stage is to fuse them with the question to predict the answer. (Chen et al., 2017) use a TF-IDF-based Document Retriever to locate relevant paragraphs from Wikipedia documents, and then use a Document Reader to predict the start and end positions of the answer span. (Lin et al., 2018) introduce an additional module of Paragraph Selector to remove noised paragraphs that explicitly do not contain an answer. (Das et al., 2019) employ a multi-step reasoner to build a new query, then rerank …

Content truncated to 500 words