Abstains from Prediction: Towards Robust Relation Extraction in Real World

Abstains from Prediction: Towards Robust Relation Extraction in Real World Jun Zhao1∗, Yongxin Zhang1∗, Nuo Xu1, Tao Gui1†, Qi Zhang1†, Yunwen Chen2, Xiang Gao2 1 School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, China 2 DataGrand Information Technology (Shanghai) Co., Ltd., Shanghai, China {zhaoj19,yongxinzhang20,tgui,qz}@fudan.edu.cn xun22@m.fudan.edu.cn {chenyunwen,gaoxiang}@datagrand.com Abstract Supervised learning is a classic paradigm of relation extraction (RE). However, a well-performing model can still confidently make arbitrarily wrong predictions when exposed to samples of unseen relations. In this work, we propose a relation extraction method with rejection option to improve robustness to unseen relations. To enable the classifier to reject unseen relations, we introduce contrastive learning techniques and carefully design a set of class-preserving transformations to improve the discriminability between known and unseen relations. Based on the learned representation, inputs of unseen relations are assigned a low confidence score and rejected. Off-the-shelf open relation extraction (OpenRE) methods can be adopted to discover the potential relations in these rejected inputs. In addition, we find that the rejection can be further improved via readily available distantly supervised data. Experiments on two public datasets prove the effectiveness of our method capturing discriminative representations for unseen relation rejection. 1 Introduction Relation extraction aims to predict the relation between entities based on their context. The extracted relational facts play a vital role in various natural language processing applications, such as knowledge base enrichment (Distiawan et al., 2019), web search (Xiong et al., 2017), and question answering (Honovich et al., 2021). To improve the quality of extracted relational facts and benefit downstream tasks, many efforts have been devoted to this task. Supervised relation extraction is a representative paradigm built upon the closed world assumption (Gallaire and Minker, 1978). Benefiting from artfully designed network architectures (Miwa and Bansal, 2016; Huang and Wang, 2017; Zhang et al., 2018) and valuable knowledge in pretrained language model (Du et al., 2018; Verga et al., 2018; Wu and He, 2019; Baldini Soares et al., 2019), models effectively capture semantic-rich representations and achieves superior results. However, conventional supervised relation extraction suffer from the lack of large-scale labeled data. To tackle this issue, distantly supervised relation extraction has attracted much attention. The existing works mainly focus on how to alleviate the noise generated in the automatic annotation. Common approaches include selecting informative instances (Lin et al., 2016), incorporating extra information (Zhang et al., 2019), and designing sophisticated training (Ma et al., 2021). Although a supervised relation classifier achieves excellent performance on known relations, real- world inputs are often mixed with samples of unseen relations. A well-performing model can still confidently make arbitrarily wrong predictions when dealing with these unseen relations (Nguyen et al., 2014; Recht et al., 2019). The unrobustness is rooted in the Shortcut feature (Geirhos et al., 2020) of neural networks. Models optimized by a supervised objective does not actively learn features beyond the bare minimum necessary to discriminate between known relations. As shown in Figure 1, if there is only president relation in the training …

Content truncated to 500 words