An Enhanced Knowledge Injection Model for Commonsense Generation

Proceedings of the 28th International Conference on Computational Linguistics, pages 2014–2025 Barcelona, Spain (Online), December 8-13, 2020 2014 An Enhanced Knowledge Injection Model for Commonsense Generation Zhihao Fan1∗, Yeyun Gong2, Zhongyu Wei1,5†, Siyuan Wang1, Yameng Huang3, Jian Jiao3, Xuanjing Huang4, Nan Duan2, Ruofei Zhang3 1School of Data Science, Fudan University, China 2Microsoft Research Asia, 3Microsoft 4School of Computer Science, Fudan University, China 5Research Institute of Intelligent and Complex Systems, Fudan University, China {fanzh18,zywei,wangsy18,xjhuang}@fudan.edu.cn, {yegong,yameng.huang,Jian.Jiao,nanduan,bzhang}@microsoft.com Abstract Commonsense generation aims at generating plausible everyday scenario description by means of reasoning about the concept combination. Digging the relationship of concepts from scratch does not suffices to build a reasonable scene, thus we argue editing the retrieved prototype from external knowledge corpus would benefit to discriminate the priority of different concept combination and complete the scenario with introducing additional concepts. We propose to use two kind of corpus as out of domain and in domain external knowledge to retrieve the prototypes respectively. To better model the prototypes, we design two attention mechanisms to enhance the knowledge injection procedure. We conduct experiment on CommonGen benchmark, experimental results show that our method significantly improves the performance on all the metrics. 1 Introduction Recently, commonsense reasoning tasks such as SWAG (Zellers et al., 2018), CommonsenseQA (Talmor et al., 2018) and CommonGen (Lin et al., 2019b) are presented to investigate the model’s ability to make acceptable and logical assumptions about ordinary scenes in our daily life. SWAG requires to infer the probable subsequent event based on the given textual description of an event. CommonsenseQA focuses on commonsense question answering that collects commonsense questions at scale by describing the relation between concepts from CONCEPTNET. Different from these discriminative tasks, CommonGen is a generation task that not only needs to use background commonsense knowledge to conduct relational reasoning, but also compositional based generation capability. Considering CommonGen requires model to reason about a potentially infinite number of novel combinations of concepts, there is a big gap between human performance and automatic generation models even current most powerful pretrained models (Lewis et al., 2019; Dong et al., 2019; Yan et al., 2020). This demonstrates that dig the combination relationship between these concepts from scratch does not suffice for generative commonsense reasoning. Hashimoto et al. (2018) shows that the task of generating complex outputs through editing existing outputs can be easier than generating complex outputs from scratch. This inspires us to use the retrieve-and-edit framework to retrieve a prototype with these concepts to enhance the commonsense generation task. There are two main advantages of the retrieve-and-edit framework. First, the knowledge of these concepts are implicit and compositional which makes it hard to find out a plausible scene from the concepts combination, such as “feel” and “pierce” appear in the generation of BART in Table 1, the retrieved prototype sentence from external knowledge corpus would provide a relevant scenario. On the basis of the scenario bias, it would be easier for us to identify the priority of these concepts combination. Second, these concepts in …

Content truncated to 500 words