Causality-aware Concept Extraction based on Knowledge-guided Prompting

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9255–9272 July 9-14, 2023 ©2023 Association for Computational Linguistics Causality-aware Concept Extraction based on Knowledge-guided Prompting Siyu Yuan♡, Deqing Yang♡∗, Jinxi Liu♡, Shuyu Tian♡, Jiaqing Liang♡, Yanghua Xiao♠♣∗, Rui Xie♢ ♡School of Data Science, Fudan University, Shanghai, China ♠Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University ♣Fudan-Aishu Cognitive Intelligence Joint Research Center, ♢Meituan, Beijing, China {syyuan21,jxliu22, sytian21}@m.fudan.edu.cn, {yangdeqing,liangjiaqing,shawyh}@fudan.edu.cn, rui.xie@meituan.com Abstract Concepts benefit natural language understand- ing but are far from complete in existing knowl- edge graphs (KGs). Recently, pre-trained lan- guage models (PLMs) have been widely used in text-based concept extraction (CE). How- ever, PLMs tend to mine the co-occurrence as- sociations from massive corpus as pre-trained knowledge rather than the real causal effect between tokens. As a result, the pre-trained knowledge confounds PLMs to extract biased concepts based on spurious co-occurrence cor- relations, inevitably resulting in low precision. In this paper, through the lens of a Structural Causal Model (SCM), we propose equipping the PLM-based extractor with a knowledge- guided prompt as an intervention to alleviate concept bias. The prompt adopts the topic of the given entity from the existing knowledge in KGs to mitigate the spurious co-occurrence correlations between entities and biased con- cepts. Our extensive experiments on repre- sentative multilingual KG datasets justify that our proposed prompt can effectively allevi- ate concept bias and improve the performance of PLM-based CE models. The code has been released on https://github.com/ siyuyuan/KPCE. 1 Introduction The concepts in knowledge graphs (KGs) enable machines to understand natural languages better, and thus benefit many downstream tasks, such as question answering (Han et al., 2020), common- sense reasoning (Zhong et al., 2021) and entity typ- ing (Yuan et al., 2022). However, the concepts, es- pecially the fine-grained ones, in existing KGs still need to be completed. For example, in the widely used Chinese KG CN-DBpedia (Xu et al., 2017), there are nearly 17 million entities but only 0.27 million concepts in total, and more than 20% enti- ties even have no concepts. Although Probase (Wu ∗Corresponding authors. Abstract of Louisa May Alcott Louisa May Alcott was an American novelist, short story writer and poet best known as the author of the novel Little Women and its sequels Little Men and Jo's Boys. PLM-based Extraction Models Biased Concept Correct Concept Louisa May Alcott is not a novel! Figure 1: The example of concept bias. The PLM-based CE models are biased to extract novel mistakenly as the concept of Louisa May Alcott from the text. et al., 2012) is a large-scale English KG, the fine- grained concepts with two or more modifiers in it only account for 30% (Li et al., 2021). We focus on extracting multi-grained concepts from texts to complete existing KGs. Most of the existing text-based concept acquisi- tion approaches adopt the extraction scheme, which can be divided into two categories: 1) pattern- matching approaches (Auer et al., 2007; Wu et al., …

Content truncated to 500 words