Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9890–9908 July 9-14, 2023 ©2023 Association for Computational Linguistics Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge Jiangjie Chen♠, Wei Shi♠, Ziquan Fu♡∗, Sijie Cheng♠, Lei Li♣, Yanghua Xiao♠♢† ♠Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University ♡System Inc. ♣University of California, Santa Barbara ♢Fudan-Aishu Cognitive Intelligence Joint Research Center {jjchen19, sjcheng20, shawyh}@fudan.edu.cn wshi22@m.fudan.edu.cn, frank@system.com, leili@cs.ucsb.edu Abstract Large language models (LLMs) have been widely studied for their ability to store and utilize positive knowledge. However, nega- tive knowledge, such as “lions don’t live in the ocean”, is also ubiquitous in the world but rarely mentioned explicitly in the text. What do LLMs know about negative knowledge? This work examines the ability of LLMs to nega- tive commonsense knowledge. We design a constrained keywords-to-sentence generation task (CG) and a Boolean question-answering task (QA) to probe LLMs. Our experiments reveal that LLMs frequently fail to generate valid sentences grounded in negative common- sense knowledge, yet they can correctly answer polar yes-or-no questions. We term this phe- nomenon the belief conflict of LLMs. Our fur- ther analysis shows that statistical shortcuts and negation reporting bias from language model- ing pre-training cause this conflict.1 1 Introduction Most of the world knowledge exists in a positive and affirmative form (Molnar, 2000; Barker and Jago, 2012; Vrandeˇci´c and Krötzsch, 2014; Speer et al., 2017). As a result, large language models (LLMs) pre-trained on a colossal amount of texts, such as GPT-3 (Brown et al., 2020; Ouyang et al., 2022) and PaLM (Chowdhery et al., 2022), have demonstrated their remarkable abilities for storing and utilizing positive knowledge in downstream tasks. In contrast, negative knowledge, such as the commonsense statement that “lions do not live in the ocean”, is rarely mentioned in the textual world (Hossain et al., 2022).2 Such negative knowl- edge also exists in the real world, and is important ∗Work done while at Brain Technologies, Inc. †Corresponding author. 1Resources of this paper are available at https:// github.com/jiangjiechen/uncommongen. 2Hossain et al. (2022) report that sentences with negation hold up to 14.5% in the CommonsenseQA dataset (Talmor et al., 2019), 8.7% in QNLI (Rajpurkar et al., 2016), and 22.6-29.9% in general-purposed texts. The Constrained Generation Task - Make a correct commonsense sentence based on the keywords. The Question Answering Task - Answer the commonsense question. LLM LLM Question Do lions live in the ocean? Keywords lion, located at, ocean Answer No Sentence Lions live in the ocean. Conflict? Figure 1: An example of the probing tasks studied in this paper. For the same negative commonsense knowl- edge <lion, located at, ocean> which is false, we find LLMs often fail to generate texts grounded in such neg- ative knowledge while knowing its validity according to question answering. for cognitive skills such as knowing what is not true or what not to think (MacDonald, 1965; Minsky, 1997; Barker and Jago, 2012). Therefore, …

Content truncated to 500 words