Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15960–15973 July 9-14, 2023 ©2023 Association for Computational Linguistics Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction Qi Sun1,2, Kun Huang1, Xiaocui Yang2,3, Pengfei Hong2, Kun Zhang1∗, and Soujanya Poria2∗ 1Nanjing University of Science and Technology 2Singapore University of Technology and Design, 3Northeastern University {319106003718, huangkun, zhangkun}@njust.edu.cn, {pengfei_hong, sporia}@sutd.edu.sg, yangxiaocui@stumail.neu.edu.cn Abstract Document-level relation extraction (DocRE) aims to infer complex semantic relations among entities in a document. Distant supervision (DS) is able to generate massive auto-labeled data, which can improve DocRE performance. Recent works leverage pseudo labels generated by the pre-denoising model to reduce noise in DS data. However, unreliable pseudo labels bring new noise, e.g., adding false pseudo la- bels and losing correct DS labels. Therefore, how to select effective pseudo labels to denoise DS data is still a challenge in document-level distant relation extraction. To tackle this issue, we introduce uncertainty estimation technol- ogy to determine whether pseudo labels can be trusted. In this work, we propose a Document- level distant Relation Extraction framework with Uncertainty Guided label denoising, UG- DRE. Specifically, we propose a novel instance- level uncertainty estimation method, which measures the reliability of the pseudo labels with overlapping relations. By further consider- ing the long-tail problem, we design dynamic uncertainty thresholds for different types of re- lations to filter high-uncertainty pseudo labels. We conduct experiments on two public datasets. Our framework outperforms strong baselines by 1.91 F1 and 2.28 Ign F1 on the RE-DocRED dataset. 1 1 Introduction Document-level Relation Extraction (DocRE) aims to extract relations among entities in a document. In contrast to the conventional RE task that mainly focuses on sentence-level (Zhou et al., 2016; Guo et al., 2019; Tian et al., 2021), DocRE is more chal- lenging due to the complex semantic scenarios, dis- course structure of the document, and long-distant interactions between entities. To understand complex inter-sentence entity re- lations, most existing methods employ transformer- ∗Corresponding author 1https://github.com/QiSun123/UGDRE [1] Mario Bonnard ( 24 December 1889 - 22 March 1965 ) was an Italian actor and film director. [2] Bonnard was born and died in Rome. ... [11] One of his last film was The Last Days of Pompeii ( 1959 ). [12] He had to leave production early due to an illness, so the film had to be completed by Sergio Leone ... Mario Bonnard Rome 22 March 1965 The Last Days of Pompeii 24 December 1889 date of birth DS Labels Pseudo Labels with UE A DS document Sergio Leone date of death director screenwriter director place of birth place of death Mario Bonnard Rome 22 March 1965 The Last Days of Pompeii 24 December 1889 date of birth Sergio Leone date of death director screenwriter director place of birth place of death DS labels Correct Positive False Positive composer Correct Negative False Negative (0.31) (0.68) (0.28) (0.29) (0.30) (0.35) ( . ) UE Scores Figure 1: An example of the DS document. …

Content truncated to 500 words