Open Set Relation Extraction via Unknown-Aware Training

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9453–9467 July 9-14, 2023 ©2023 Association for Computational Linguistics Open Set Relation Extraction via Unknown-Aware Training Jun Zhao1∗, Xin Zhao1∗, Wenyu Zhan1, Qi Zhang1, Tao Gui2†, Zhongyu Wei3, Yunwen Chen4, Xiang Gao4, Xuanjing Huang1,5† 1School of Computer Science, Fudan University 2Institute of Modern Languages and Linguistics, Fudan University 3School of Data Science, Fudan University 4DataGrand Information Technology (Shanghai) Co., Ltd. 5International Human Phenome Institutes (Shanghai) {zhaoj19,qz,tgui}@fudan.edu.cn, zhaoxin21@m.fudan.edu.cn Abstract The existing supervised relation extraction methods have achieved impressive perfor- mance in a closed-set setting, where the rela- tions during both training and testing remain the same. In a more realistic open-set setting, unknown relations may appear in the test set. Due to the lack of supervision signals from unknown relations, a well-performing closed- set relation extractor can still confidently misclassify them into known relations. In this paper, we propose an unknown-aware training method, regularizing the model by dynamically synthesizing negative instances. To facilitate a compact decision boundary, “difficult” negative instances are necessary. Inspired by text adversarial attacks, we adaptively apply small but critical perturbations to original training instances and thus synthesizing negative in- stances that are more likely to be mistaken by the model as known relations. Experimental results show that this method achieves SOTA unknown relation detection without compro- mising the classification of known relations. 1 Introduction Relation extraction (RE) is an important basic task in the field of natural language processing, aiming to extract the relation between entity pairs from unstructured text. The extracted relation facts have a great practical interest to various downstream applications, such as dialog system (Madotto et al., 2018), knowledge graph (Lin et al., 2015), web search (Xiong et al., 2017), among others. Many efforts have been devoted to improving the quality of extracted relation facts (Han et al., 2020). Conventional supervised relation extraction is oriented to known relations with pre-specified schema. Hence, the paradigm follows a closed- set setting, meaning that during both training and testing the relations remain the same. Nowadays, *Equal Contributions. †Corresponding authors. (a) Supervised decision boundary : Known Relational Data (b) Expected decision boundary : Unknown Relational Data : Synthesized Negative Data Figure 1: The decision boundary optimized only on the known relations cannot cope with an open set setting, in which the input may come from the relations unobserved in training. We target at regularizing the decision boundary by synthesizing difficult negative instances. neural RE methods have achieved remarkable success within this setting (Wang et al., 2016; Wu and He, 2019); and in contrast, open relation extraction (OpenRE) is focused on discovering constantly emerging unknown relations. Common practices include directly tagging the relational phrases that link entity pairs (Zhan and Zhao, 2020), and clustering instances with the same relation (Hu et al., 2020; Zhao et al., 2021). However, relation extraction in real applications follows an open-set setting, meaning that both known and unknown relations are mixed within testing data.* This requires that …

Content truncated to 500 words