Differentiable Open-Ended Commonsense Reasoning

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4611–4625 June 6–11, 2021. ©2021 Association for Computational Linguistics 4611 Differentiable Open-Ended Commonsense Reasoning Bill Yuchen Lin1∗, Haitian Sun2, Bhuwan Dhingra2, Manzil Zaheer2, Xiang Ren1, William W. Cohen2 1 University of Southern California 2 Google Research {yuchen.lin,xiangren}@usc.edu {haitiansun,bdhingra,manzilzaheer,wcohen}@google.com Abstract Current commonsense reasoning research fo- cuses on developing models that use common- sense knowledge to answer multiple-choice questions. However, systems designed to an- swer multiple-choice questions may not be useful in applications that do not provide a small list of candidate answers to choose from. As a step towards making commonsense rea- soning research more realistic and useful, we propose to study open-ended commonsense reasoning (OpenCSR) — the task of answer- ing a commonsense question without any pre- defined choices — using as a resource only a knowledge corpus of commonsense facts writ- ten in natural language. OpenCSR is challeng- ing due to a large decision space, and because many questions require implicit multi-hop rea- soning. As an approach to OpenCSR, we propose DRFACT, an efficient Differentiable model for multi-hop Reasoning over knowl- edge Facts. To evaluate OpenCSR meth- ods, we adapt three popular multiple-choice datasets, and collect multiple new answers to each test question via crowd-sourcing. Exper- iments show that DRFACT outperforms strong baseline methods by a large margin.1 1 Introduction The conventional task setting for most current commonsense reasoning research is multiple- choice question answering (QA) — i.e., given a question and a small set of pre-defined an- swer choices, models are required to determine which of the candidate choices best answers the question. Existing commonsense reasoning mod- els usually work by scoring a question-candidate pair (Lin et al., 2019; Lv et al., 2020; Feng et al., 2020). Hence, even an accurate multiple-choice ∗ The work was mainly done during Bill Yuchen Lin’s internship at Google Research. 1Our code and data are available at the project website — https://open-csr.github.io/. The human anno- tations were collected by the USC-INK group. carbon dioxide is the major greenhouse gas contributing to global warming . trees remove carbon dioxide from the atmosphere through photosynthesis . Multiple-Choice CSR (reason w/ question+choice) Open-Ended CSR (reason w/ question) a large text corpus of commonsense facts …, renewable energy, tree, solar battery, … Output: a ranked list of concepts as answers. Q: What can help alleviate global warming? (A) air cooler (B) fossil fuel (C) renewable energy (D) water Multi-Hop Reasoning Figure 1: We study the task of open-ended com- monsense reasoning (OpenCSR), where answer candi- dates are not provided (as in a multiple-choice setting). Given a question, a reasoner uses multi-hop reasoning over a knowledge corpus of facts, and outputs a ranked list of concepts from the corpus. QA model cannot be directly applied in practical applications where answer candidates are not pro- vided (e.g., answering a question asked on a search engine, or during conversation with a chat-bot). Because we seek to advance commonsense rea- …

Content truncated to 500 words