Improving Distantly Supervised Relation Extraction with Self-Ensemble Noise Filtering

Proceedings of Recent Advances in Natural Language Processing, pages 1031–1039 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_116 1031 Improving Distantly Supervised Relation Extraction with Self-Ensemble Noise Filtering Tapas Nayak IIT Kharagpur India Navonil Majumder SUTD Singapore {tnk02.05,n.majumder.2009,soujanya.poria}@gmail.com Soujanya Poria SUTD Singapore Abstract Distantly supervised models are very popular for relation extraction since we can obtain a large amount of training data using the distant supervision method without human annotation. In distant supervision, a sentence is considered as a source of a tuple if the sentence contains both entities of the tuple. However, this con- dition is too permissive and does not guaran- tee the presence of relevant relation-specific information in the sentence. As such, dis- tantly supervised training data contains much noise which adversely affects the performance of the models. In this paper, we propose a self- ensemble filtering mechanism to filter out the noisy samples during the training process. We evaluate our proposed framework on the New York Times dataset which is obtained via dis- tant supervision. Our experiments with mul- tiple state-of-the-art neural relation extraction models show that our proposed filtering mech- anism improves the robustness of the models and increases their F1 scores. 1 Introduction The task of relation extraction is about finding rela- tion or no relation between two entities. This is an important task to fill the gaps of existing knowledge bases (KB). Open information extraction (OpenIE) (Banko et al., 2007) is one way of extracting rela- tions from text. They consider the verb in a sen- tence as the relation and then find the noun phrases located to the left and right of that verb as the en- tities. But this process has two serious problems: First, the same relation can appear in the text with many verb forms and OpenIE treats them as dif- ferent relations. This leads to the duplication of relations in KB. Second, OpenIE treats any verbs in a sentence as a relation which can generate a large number of insignificant tuples which cannot be added to a KB. Supervised relation extraction models, on the other hand, do not have these problems. But they require a large amount of annotated data which is difficult to get. Mintz et al. (2009), Riedel et al. (2010), and Hoffmann et al. (2011) used the idea of distant supervision to automatically obtain the training data to overcome this problem. The idea of distant supervision is that if a sentence contains both the entities of a tuple, it is chosen as a source sentence of this tuple. Although this process can generate some noisy training instances, it can give a significant amount of training data which can be used to build supervised models for this task. They map the tuples from existing KBs such as Freebase (Bollacker et al., 2008) to the text corpus such as Wikipedia articles (Mintz et al., 2009) or New York Times articles (Riedel et al., 2010; Hoffmann et al., 2011). Based on distantly supervised training data, re- searchers have proposed many state-of-the-art mod- …

Content truncated to 500 words