From the One, Judge of the Whole: Typed Entailment Graph Construction with Predicate Generation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3534–3551 July 9-14, 2023 ©2023 Association for Computational Linguistics From the One, Judge of the Whole: Typed Entailment Graph Construction with Predicate Generation Zhibin Chen123 Yansong Feng13∗ Dongyan Zhao123 1Wangxuan Institute of Computer Technology, Peking University, China 2Center for Data Science, Peking University, China 3The MOE Key Laboratory of Computational Linguistics, Peking University, China {czb-peking, fengyansong, zhaody}@pku.edu.cn Abstract Entailment Graphs (EGs) have been con- structed based on extracted corpora as a strong and explainable form to indicate context- independent entailment relations in natural lan- guages. However, EGs built by previous meth- ods often suffer from the severe sparsity issues, due to limited corpora available and the long- tail phenomenon of predicate distributions. In this paper, we propose a multi-stage method, Typed Predicate-Entailment Graph Generator (TP-EGG), to tackle this problem. Given sev- eral seed predicates, TP-EGG builds the graphs by generating new predicates and detecting en- tailment relations among them. The generative nature of TP-EGG helps us leverage the recent advances from large pretrained language mod- els (PLMs), while avoiding the reliance on care- fully prepared corpora. Experiments on bench- mark datasets show that TP-EGG can generate high-quality and scale-controllable entailment graphs, achieving significant in-domain im- provement over state-of-the-art EGs and boost- ing the performance of down-stream inference tasks1. 1 Introduction The entailment relation between textual predicates plays a critical role in natural language infer- ence and natural language understanding tasks, in- cluding question answering (Pathak et al., 2021; McKenna et al., 2021) and knowledge graph com- pletion (Yoshikawa et al., 2019; Hosseini et al., 2019, 2021). To detect entailment relations, previ- ous works pay attention to the Recognizing Textual Entailment (RTE) task, which takes a pair of sen- tences as input and predicts whether one sentence entails the other (Bowman et al., 2015; He et al., 2021b; Pilault et al., 2020). Current RTE mod- els perform well on RTE benchmarks, but most of ∗Corresponding author. 1Our code is available at https://github.com/ ZacharyChenpk/TP-EGG A adore B A recognize B A know B A=person B=government Figure 1: An example of typed entailment graph includ- ing several predicates, with argument types person and government. them are lacking in explainability, as they make use of the black-box Language Models (LM) without providing any explainable clues. Recent works focus on learning the Entailment Graph (EG) structure, which organizes typed predi- cates in directional graphs with entailment relations as the edges (Hosseini et al., 2018, 2019; McKenna et al., 2021), as shown in Figure 1. With the explicit graph structure containing predicates and their en- tailment relations, similar to Knowledge Graphs (KGs), using EGs becomes an explainable and context-independent way to represent the knowl- edge required in natural language inference and other NLP tasks. Most existing EGs are constructed with the Dis- tributional Inclusion Hypothesis (DIH), which sug- gests that all typical context features of a pred- icate v can also occur with another predicate w if v entails …

Content truncated to 500 words