CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6191–6200 August 1–6, 2021. ©2021 Association for Computational Linguistics 6191 CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction Tao Chen1,2, Haizhou Shi1, Siliang Tang1,2∗, Zhigang Chen3, Fei Wu1 & Yueting Zhuang1 1Zhejiang University 2Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies 3State Key Laboratory of Cognitive Intelligence, Hefei, China {ttc, shihaizhou, siliang, yzhuang}@zju.edu.cn zgchen@iflytek.com, wufei@cs.zju.edu.cn Abstract The journey of reducing noise from distant supervision (DS) generated training data has been started since the DS was first introduced into the relation extraction (RE) task. For the past decade, researchers apply the multi- instance learning (MIL) framework to find the most reliable feature from a bag of sentences. Although the pattern of MIL bags can greatly reduce DS noise, it fails to represent many other useful sentence features in the datasets. In many cases, these sentence features can only be acquired by extra sentence-level hu- man annotation with heavy costs. Therefore, the performance of distantly supervised RE models is bounded. In this paper, we go beyond typical MIL framework and propose a novel Contrastive Instance Learning (CIL) framework. Specifically, we regard the ini- tial MIL as the relational triple encoder and constraint positive pairs against negative pairs for each instance. Experiments demonstrate the effectiveness of our proposed framework, with significant improvements over the previ- ous methods on NYT10, GDS and KBP. 1 Introduction Relation extraction (RE) aims at predicting the re- lation between entities based on their context. Sev- eral studies have been carried out to handle this crucial and complicated task over decades as the extracted information can serve as a significant role for many downstream tasks. Since the amount of training data generally limits traditional supervised RE systems, current RE systems usually resort to distant supervision (DS) to fetch abundant train- ing data by aligning knowledge bases (KBs) and texts. However, such a heuristic way inevitably in- troduces some noise to the generated data. Training a robust and unbiased RE system under DS data ∗ Corresponding author � �� �� �� �� �� �� � � �� �� �� �� �� �� �� � � � �� �� �� �� �� �� Sentence Encoder �� � �� �� �� �� �� � �� �� �� � KB Fact [��, ��, �] � Figure 1: Classical MIL framework for DSRE. (Left) A set of instances (x1, x2, . . . , xm) with the same KB fact [e1, e2, r] form a bag B; (Right) The MIL frame- work trains the DSRE model at bag level( �B : � αihi). noise becomes the biggest challenge for distantly supervised relation extraction (DSRE). With awareness of the existing DS noise, Zeng et al. (2015) introduces the multi-instance learning (MIL) framework to DSRE by dividing training instances into several bags and using bags as new data units. Regarding the strategy for selecting in- stances inside the bag, the soft …

Content truncated to 500 words