NegatER: Unsupervised Discovery of Negatives in Commonsense Knowledge Bases

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5633–5646 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5633 NEGATER: Unsupervised Discovery of Negatives in Commonsense Knowledge Bases Tara Safavi, Jing Zhu, Danai Koutra University of Michigan, Ann Arbor {tsafavi,jingzhuu,dkoutra}@umich.edu Abstract Codifying commonsense knowledge in ma- chines is a longstanding goal of artificial in- telligence. Recently, much progress toward this goal has been made with automatic knowl- edge base (KB) construction techniques. How- ever, such techniques focus primarily on the acquisition of positive (true) KB statements, even though negative (false) statements are often also important for discriminative rea- soning over commonsense KBs. As a first step toward the latter, this paper proposes NegatER, a framework that ranks potential negatives in commonsense KBs using a con- textual language model (LM). Importantly, as most KBs do not contain negatives, NegatER relies only on the positive knowledge in the LM and does not require ground-truth nega- tive examples. Experiments demonstrate that, compared to multiple contrastive data aug- mentation approaches, NegatER yields neg- atives that are more grammatical, coherent, and informative—leading to statistically sig- nificant accuracy improvements in a challeng- ing KB completion task and confirming that the positive knowledge in LMs can be “re- purposed” to generate negative knowledge. 1 Introduction Endowing machines with commonsense, which is knowledge that members of a culture usually agree upon but do not express explicitly, is a ma- jor but elusive goal of artificial intelligence (Min- sky, 1974; Davis et al., 1993; Liu and Singh, 2004; Davis and Marcus, 2015). One way to capture such knowledge is with curated commonsense knowl- edge bases (KBs), which contain semi-structured statements of “everyday” human knowledge. As such KBs are increasingly being used to augment the capabilities of intelligent agents (Hwang et al., 2021), automatically expanding their scope has be- come crucial (Li et al., 2016; Davison et al., 2019; Bosselut et al., 2019; Malaviya et al., 2020). Table 1: Out-of-KB statements are less meaningful as negative examples when sampled at random versus ranked with our NEGATER framework. The random examples are taken from the test split of the Concept- Net benchmark introduced by Li et al. (2016). Method Negative statement Random sampling (“tickle”, HASSUBEVENT, “supermarket”) (“lawn mower”, ATLOCATION, “pantry”) (“closet”, USEDFOR, “play baseball”) NEGATER ranking (“ride horse”, HASSUBEVENT, “pedal”) (“zoo keeper”, ATLOCATION, “jungle”) (“air ticket”, USEDFOR, “get onto trolley”) Previous research in this direction focuses pri- marily on the acquisition of positive knowledge, or that which is true about the world. However, understanding what is true about the world often also requires gathering and reasoning over explic- itly untrue information. Humans routinely rely on negative knowledge—that is, what “not to do” or what “not to believe”—in order to increase cer- tainty in decision-making and avoid mistakes and accidents (Minsky, 1994). Similarly, discrimina- tive models that operate over structured knowledge from KBs often require explicit negative examples in order to learn good decision boundaries (Sun et al., 2019; Ahrabian et al., 2020; Ma et al., 2021). The …

Content truncated to 500 words