NS-Hunter: BERT-Cloze Based Semantic Denoising for Distantly Supervised Relation Classification

NS-Hunter: BERT-Cloze Based Semantic Denoising for Distantly Supervised Relation Classification Tielin Shen, Daling Wang, Shi Feng, Yifei Zhang School of Computer Science and Engineering, Northeastern University, Shenyang, China {shentielin, wangdaling, fengshi, zhangyifei}@cse.neu.edu.cn Abstract Distant supervision can generate large-scale relation classification data quickly and economi- cally. However, a great number of noise sentences are introduced which can not express their labeled relations. By means of pre-trained language model BERT’s powerful function, in this paper, we propose a BERT-based semantic denoising approach for distantly supervised relation classification. In detail, we define an entity pair as a source entity and a target entity. For the specific sentences whose target entities in BERT-vocabulary (one-token word), we present the differences of dependency between two entities for noise and non-noise sentences. For general sentences whose target entity is multi-token word, we further present the differences of last hid- den states of [MASK]-entity (MASK-lhs for short) in BERT for noise and non-noise sentences. We regard the dependency and MASK-lhs in BERT as two semantic features of sentences. With BERT, we capture the dependency feature to discriminate specific sentences first, then capture the MASK-lhs feature to denoise distant supervision datasets. We propose NS-Hunter, a novel denoising model which leverages BERT-cloze ability to capture the two semantic features and integrates above functions. According to the experiment on NYT data, our NS-Hunter model achieves the best results in distant supervision denoising and sentence-level relation classifica- tion. Keywords: Distant supervision, relation classification, semantic denoising 1 Introduction Relation classification (RC) is a fundamental task in natural language processing. The goal of RC (Ze- lenko et al., 2003) is to identify the relation type in a sentence for a given entity pair, which is particularly important for the construction of knowledge bases. In recent years, deep learning has performed very well in relation extraction, but the technique needs a great number of labeled data, which is very expen- sive for manual tagging. In order to obtain a large amount of labeled RC data quickly and cheaply, distant supervision (DS) (Mintz et al., 2009) was proposed to automatically generate data by aligning a knowl- edge base with an unlabeled corpus. It is built on a weak assumption that if an entity pair has a relation in a knowledge base, all the sentences containing this entity pair will express the corresponding relation and exist in the dataset as a bag (Mintz et al., 2009). Based on such an assumption, a large number of noise sentences are generated by DS because many sentences can not express their labeled relation in fact. For example, in Figure 1, sentence S-2 can not express the labeled relation “founder”. These noise sentences such as S-2 will cause error propagation and may significantly reduce the performance of RC model. Since the pre-trained language model BERT was put forward (Devlin et al., 2018), it has performed very well in the fully supervised RC datasets such as SemEval 2010 task 8 (Soares et al., 2019; Wu and He, 2019). Different from DS, it …

Content truncated to 500 words