CN-AutoMIC: Distilling Chinese Commonsense Knowledge from Pretrained Language Models

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9253 - 9265 December 7-11, 2022 ©2022 Association for Computational Linguistics CN-AutoMIC: Distilling Chinese Commonsense Knowledge from Pretrained Language Models Chenhao Wang1,2, Jiachun Li1,2, Yubo Chen1,2, Kang Liu1,2,3 and Jun Zhao1,2 1National Laboratory of Pattern Recognition, Institute of Automation, CAS, Beijing, China 2School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China 3Beijing Academy of Artificial Intelligence, Beijing, China {chenhao.wang,jiachun.li,yubo.chen,kliu,jzhao}@nlpr.ia.ac.cn Abstract Commonsense knowledge graphs (CKGs) are increasingly applied in various natural lan- guage processing tasks. However, most exist- ing CKGs are limited to English, which hin- ders related research in non-English languages. Meanwhile, directly generating commonsense knowledge from pretrained language models has recently received attention, yet it has not been explored in non-English languages. In this paper, we propose a large-scale Chinese CKG generated from multilingual PLMs, named as CN-AutoMIC, aiming to fill the research gap of non-English CKGs. To improve the effi- ciency, we propose generate-by-category strat- egy to reduce invalid generation. To ensure the filtering quality, we develop cascaded filters to discard low-quality results. To further increase the diversity and density, we introduce a boot- strapping iteration process to reuse generated results. Finally, we conduct detailed analyses on CN-AutoMIC from different aspects. Em- pirical results show the proposed CKG has high quality and diversity, surpassing the direct trans- lation version of similar English CKGs. We also find some interesting deficiency patterns and differences between relations, which reveal pending problems in commonsense knowledge generation. We share the resources and related models for further study1. 1 Introduction Compiling large-scale commonsense knowledge resources is a long-term goal of the AI commu- nity. Recent efforts focus on constructing com- monsense knowledge graphs (CKGs) via manually compiling (Speer et al., 2017; Sap et al., 2019; Mostafazadeh et al., 2020) or automatic extrac- tion (Tandon et al., 2014; Romero et al., 2019; Zhang et al., 2020; Nguyen et al., 2021). These projects have shown benefits for a wide range of downstream tasks (Lin et al., 2019; Tian et al., 2020; Ammanabrolu et al., 2021). 1http://github.com/wchrepo/cnautomic/ However, most CKG projects are limited to En- glish, which hinders further research in other lan- guages. To go beyond such an English-centric trend in commonsense knowledge research, there are some challenging issues. First, directly translating English CKGs is not enough. For example, (PersonX is a little girl, xWant, to ask Christmas presents) is a triple from a CKG crowdsourced by English users (Sap et al., 2019), but it is not common in non-Christian cul- tures. In fact, such resources reflect only the com- monsense perspectives of English-speaking com- munities. The translation will omit the cultural differences in other languages, and even implicitly exacerbate the English-centric bias (Mehrabi et al., 2021). Therefore, when creating CKGs for new languages, it is better to rely on native speakers and corpora. Second, current common practices in En- glish CKG construction, i.e. manually compiling and automatic extraction, are difficult to generalize. For manually compiling, creating human-authored CKGs for …

Content truncated to 500 words