Finding Influential Instances for Distantly Supervised Relation Extraction

Proceedings of the 29th International Conference on Computational Linguistics, pages 2639–2650 October 12–17, 2022. 2639 Finding Influential Instances for Distantly Supervised Relation Extraction Zifeng Wang1,2, Rui Wen3, Xi Chen3∗, Shao-Lun Huang2, Ningyu Zhang4, Yefeng Zheng3 1University of Illinois Urbana Champaign 2TBSI, Tsinghua University 3Tencent 4Zhejiang University Abstract Distant supervision (DS) is a strong way to ex- pand the datasets for enhancing relation extrac- tion (RE) models but often suffers from high label noise. Current works based on attention, reinforcement learning, or GAN are black-box models so they neither provide meaningful in- terpretation of sample selection in DS nor sta- bility on different domains. On the contrary, this work proposes a novel model-agnostic in- stance sampling method for DS by influence function (IF), namely REIF. Our method identi- fies favorable/unfavorable instances in the bag based on IF, then does dynamic instance sam- pling. We design a fast influence sampling algo- rithm that reduces the computational complex- ity from O(mn) to O(1), with analyzing its ro- bustness on the selected sampling function. Ex- periments show that by simply sampling the fa- vorable instances during training, REIF is able to win over a series of baselines which have complicated architectures. We also demon- strate that REIF can support interpretable in- stance selection. 1 Introduction To expand the training data for relation extraction (RE), distant supervision (DS) was proposed by Mintz et al. (2009) who assumed that if two enti- ties are related in existing KBs, then all sentences contain both of them express this relation. How- ever, this heuristic inevitably suffers from wrong labels (Takamatsu et al., 2012) and undermines model performance. For example, the sentence “Bill Gates redefined the software industry, ... said Rob Glaser, a former Microsoft executive" does not mention the relation founder but is still treated as a positive training sample in DS. Dealing with noisy instances in DS has been a focus in RE. There are three main genres in the literature: (1) incorporat- ing an attention module (Lin et al., 2016) to allocate confidence level among instances in the same bag; ∗ Corresponding author: jasonxchen@tencent.com Bag Label Instances Influences 𝝓 Probability 𝝅 /business /company /founders (Bill Gates, Microsoft) Bill Gates, the founder of Microsoft, has indicates … -0.8 0.7 … as an investor, Allen, who founded Microsoft with Bill Gates, has … -0.2 0.5 Bill Gates redefined the software industry … said Rob Glaser, a former Microsoft executive… 0.1 0.2 Figure 1: Finding influential instances within a bag via subsampling based on the calculated probability π. Note that here negative ϕ means a beneficial sample. (2) using reinforcement learning (Qin et al., 2018b) for instance selection; and (3) leveraging adversar- ial training (Wu et al., 2017) to enhance the RE model’s robustness against noise. However, they are either black-box models thus unable to provide meaningful interpretation of sample selection or sensitive to datasets. More importantly, none of them is theoretically guaranteed to truely reduce the “noise” from the dataset. In this work, we propose to leverage influence …

Content truncated to 500 words