PInKS: Preconditioned Commonsense Inference with Minimal Supervision

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 320–336 November 20–23, 2022. ©2022 Association for Computational Linguistics 320 PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS PInKS: Preconditioned Commonsense Inference with Minimal Supervision Ehsan Qasemi1 and Piyush Khanna2 and Qiang Ning3 and Muhao Chen1 1University of Southern California 2Delhi Technological University 3Amazon {qasemi,muhaoche}@usc.edu; piyushkhanna_bt2k17@dtu.ac.in; qning@amazon.com Abstract Reasoning with preconditions such as “glass can be used for drinking water unless the glass is shattered” remains an open problem for language models. The main challenge lies in the scarcity of preconditions data and model’s lack of support for such reasoning. We present PInKS , Preconditioned Com- monsense Inference with WeaK Supervision, an improved model for reasoning with pre- conditions through minimum supervision. We show, both empirically and theoretically, that PInKS improves the results on benchmarks fo- cused on reasoning with the preconditions of commonsense knowledge (up to 40% Macro- F1 scores). We further investigate PInKS through PAC-Bayesian informativeness analy- sis, precision measures, and ablation study.1 1 Introduction Inferring the effect of a situation or precondition on a subsequent action or state (illustrated in Fig. 1) is an open part of commonsense reasoning. It re- quires an agent to possess and understand different dimensions of commonsense knowledge (Wood- ward, 2011), e.g. physical, causal, social, etc. This ability can improve many knowledge-driven tasks such as question answering (Wang et al., 2019; Talmor et al., 2019), machine reading comprehen- sion (Sakaguchi et al., 2020), and narrative predic- tion (Mostafazadeh et al., 2016). It also seeks to benefit a wide range of real-world intelligent appli- cations such as legal document processing (Hage, 2005), claim verification (Nie et al., 2019), and debate processing (Widmoser et al., 2021). Multiple recent studies have taken the effort on reasoning with preconditions of commonsense knowledge (Rudinger et al., 2020; Qasemi et al., 2022; Mostafazadeh et al., 2020; Hwang et al., 2020). These studies show that preconditioned rea- soning represents an unresolved challenge to state- 1Code and data on https://github.com/luka-group/PInKS Figure 1: Examples on Preconditioned Inference and the NLI format they can be represented in. of-the-art (SOTA) language model (LM) based rea- soners. Generally speaking, the problem of rea- soning with preconditions has been formulated as variations of the natural language inference (NLI) task where, given a precondition/update, the model has to decide its effect on a common sense statement or chain of statements. For exam- ple, PaCo (Qasemi et al., 2022) approaches the task from the causal (hard reasoning) perspective in term of enabling and disabling preconditions of commonsense knowledge, and evaluate reason- ers with crowdsourced commonsense statements about the two polarities of preconditions of state- ments in ConceptNet (Speer et al., 2017). Similarly, δ−NLI (Rudinger et al., 2020) formulates the prob- lem from soft assumptions’ perspective, i.e., weak- eners and strengtheners, and justifies whether the update sentence weakens or strengthens …

Content truncated to 500 words