Do Children Texts Hold The Key To Commonsense Knowledge?

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10954 - 10959 December 7-11, 2022 ©2022 Association for Computational Linguistics Do Children Texts Hold The Key To Commonsense Knowledge? Julien Romero T´el´ecom SudParis jromero@telecom-sudparis.eu Simon Razniewski Max Planck Institute for Informatics srazniew@mpi-inf.mpg.de Abstract Compiling comprehensive repositories of com- monsense knowledge is a long-standing prob- lem in AI. Many concerns revolve around the issue of reporting bias, i.e., that frequency in text sources is not a good proxy for rele- vance or truth. This paper explores whether children’s texts hold the key to commonsense knowledge compilation, based on the hypoth- esis that such content makes fewer assump- tions on the reader’s knowledge, and therefore spells out commonsense more explicitly. An analysis with several corpora shows that chil- dren’s texts indeed contain much more, and more typical commonsense assertions. More- over, experiments show that this advantage can be leveraged in popular language-model-based commonsense knowledge extraction settings, where task-unspecific fine-tuning on small amounts of children texts (childBERT) already yields significant improvements. This pro- vides a refreshing perspective different from the common trend of deriving progress from ever larger models and corpora. 1 Introduction Compiling commonsense knowledge (CSK) is a long-standing problem in AI (Lenat, 1995). Au- tomated text-extraction-based approaches to CSK compilation, like Knext (Gordon et al., 2010), TupleKB (Dalvi Mishra et al., 2017), Quasi- modo (Romero et al., 2019), COMET (Hwang et al., 2021) or Ascent (Nguyen et al., 2021) typ- ically struggle with reporting bias (Gordon and Van Durme, 2013; Mehrabi et al., 2021), in particu- lar an under-reporting of basic commonsense asser- tions. This is a crux of commonsense: If knowledge is assumed to be commonplace, such as that rain is wet or cars have wheels, there is little need to utter it explicitly. In contrast, statements that contradict commonsense are more frequently reported, lead- ing to inappropriate images of the real world, e.g., that fires are more often cold than hot (e.g., 238 vs. 173 literal occurrences in the English Wikipedia). Children’s material may partially counter this bias: As children’s knowledge is still growing, seemingly obvious assertions may still be fre- quently expressed explicitly in such material. Note that this is not a binary question of whether some knowledge is expressed or not, but more a rank- ing problem: Prominent CSK repositories often do not struggle to recall relevant statements (e.g., As- cent (Nguyen et al., 2021) contains 2800 assertions for “elephant”), but struggle to rank them properly. This is especially true for language-model based ap- proaches of CSK compilation (Hwang et al., 2021; West et al., 2022), which by design can assign every token in the vocabulary a probability, but should do so in sensible order. This paper investigates (i) whether children’s texts are a promising source for CSK and (ii) whether small corpora can still boost knowledge extraction from large language models. Specif- ically, we analyze the density and typicality of CSK assertions in children’s text corpora and show how fine-tuning existing language …

Content truncated to 500 words