A Simple Model for Distantly Supervised Relation Extraction

Proceedings of the 29th International Conference on Computational Linguistics, pages 2651–2657 October 12–17, 2022. 2651 A Simple Model for Distantly Supervised Relation Extraction Ziqin Rao, Fangxiang Feng, Ruifan Li∗, Xiaojie Wang School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China {ziqin_rao, fxfeng, rfli, xjwang}@bupt.edu.cn Abstract Distantly supervised relation extraction is chal- lenging due to the noise within data. Recent methods focus on exploiting bag representa- tions based on deep neural networks with com- plex de-noising scheme to achieve remarkable performance. In this paper, we propose a sim- ple but effective BERT-based Graph convolu- tional network Model (i.e., BGM). Our BGM comprises of an instance embedding module and a bag representation module. The instance embedding module uses a BERT-based pre- trained language model to extract key infor- mation from each instance. The bag represen- taion module constructs the corresponding bag graph then apply a convolutional operation to obtain the bag representation. Our BGM model achieves a considerable improvement on two benchmark datasets, i.e., NYT10 and GDS1. 1 Introduction In the distant supervision relation extraction (DS- RE) setting, handling the noisy training data is a major challenge for downstream applications. To alleviate the severe noise problem in DS-RE, Riedel et al. (2010) incorporate the multi-instance learning (MIL) framework. Under this framework, the instances (i.e., sentences) for an identical entity pair are regarded as a bag. However, learning ef- fective bag representations from the noisy data is a challenge resulting in unsatisfactory performance. Recently, to obtain effective bag representations, various neural models are incorporated. Lin et al. (2016) propose a selective attention mechanism to capture relatively informative instances form- ing the bag representation. Vashishth et al. (2018) use graph convolution network (GCN) (Kipf and Welling, 2017) to encode syntactic information ob- tained from a dependency parser. However, the *Corresponding author. 1Code and datasets are available at https://github. com/ziqinrao. GCN’s capacity in capturing the correlation among instances is not sufficiently explored. Meanwhile, a prevalent trend is to use pretrained language models (PLMs) for various NLP tasks (Alberti et al., 2019; Tang et al., 2021; Wang et al., 2021; Li et al., 2021). PLMs work without ex- plicit linguistic features and side-information like POS tags and entity types. Alt et al. (2019) use a PLM to incorporate more linguistic and seman- tic information. The correlations among instances are implicitly represented through a naïve attention mechanism. Very recently, Chen et al. (2021) pro- pose PLMs combined with the contrastive instance learning (CIL) to build bag representations. CIL captures the correlation among instances through data augmentation with positive and negative pairs. These PLMs-based methods achieve a fabulous performance. Hence, a question arises, how about combining PLMs and GCNs to learn the instance correlations for bag representations? Towards this goal, we propose an impressively simple model, i.e, BGM. Our BGM comprises of a PLM (e.g., BERT) and a concise GCN. The PLM brings accurate contextual representations for in- stances. The GCN whose nodes are instances en- codes the correlation of instances in a bag through mutually …

Content truncated to 500 words