Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3071 - 3081 July 10-15, 2022 ©2022 Association for Computational Linguistics Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis Yiwei Wang1 Muhao Chen2 Wenxuan Zhou 2 Yujun Cai3 Yuxuan Liang1 Dayiheng Liu4 Baosong Yang4 Juncheng Liu1 Bryan Hooi1 1 National University of Singapore 2 University of Southern California 3 Nanyang Technological University 4Alibaba Group wangyw seu@foxmail.com Abstract Recent literature focuses on utilizing the entity information in the sentence-level relation ex- traction (RE), but this risks leaking superficial and spurious clues of relations. As a result, RE still suffers from unintended entity bias, i.e., the spurious correlation between entity men- tions (names) and relations. Entity bias can mislead the RE models to extract the relations that do not exist in the text. To combat this issue, some previous work masks the entity mentions to prevent the RE models from over- fitting entity mentions. However, this strategy degrades the RE performance because it loses the semantic information of entities. In this paper, we propose the CORE (Counterfactual Analysis based Relation Extraction) debi- asing method that guides the RE models to focus on the main effects of textual context without losing the entity information. We first construct a causal graph for RE, which mod- els the dependencies between variables in RE models. Then, we propose to conduct coun- terfactual analysis on our causal graph to dis- till and mitigate the entity bias, that captures the causal effects of specific entity mentions in each instance. Note that our CORE method is model-agnostic to debias existing RE sys- tems during inference without changing their training processes. Extensive experimental re- sults demonstrate that our CORE yields signif- icant gains on both effectiveness and general- ization for RE. The source code is provided at: https://github.com/vanoracai/CoRE. 1 Introduction Sentence-level relation extraction (RE) is an impor- tant step to obtain a structural perception of unstruc- tured text (Distiawan et al., 2019) by extracting relations between entity mentions (names) from the textual context. From human oracle, textual context should be the main source of information that determines the ground-truth relations between entities. Consider a sentence “::::: Mary gave birth to Jerry.”1. Even if we change the entity mentions from ‘Jerry’ and ‘Mary’ to other people’s names, the relation ‘parents’ still holds between the sub- ject and object as described by the textual context “gave birth to”. Recently, some work aims to utilize entity men- tions for RE (Yamada et al., 2020; Zhou and Chen, 2021), which, however, leak superficial and spuri- ous clues about the relations (Zhang et al., 2018). In our work, we observe that entity information can lead to biased relation prediction by mislead- ing RE models to extract relations that do not exist in the text. Fig. 1 visualizes a relation prediction from a state-of-the-art RE model (Alt et al., 2020) (see more examples in Tab. 7). Although the con- …

Content truncated to 500 words