Structured Minimally Supervised Learning for Neural Relation Extraction

Proceedings of NAACL-HLT 2019, pages 3057–3069 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3057 Structured Minimally Supervised Learning for Neural Relation Extraction Fan Bai and Alan Ritter Department of Computer Science and Engineering The Ohio State University Columbus, OH {bai.313, ritter.1492}@osu.edu Abstract We present an approach to minimally su- pervised relation extraction that combines the benefits of learned representations and structured learning, and accurately predicts sentence-level relation mentions given only proposition-level supervision from a KB. By explicitly reasoning about missing data dur- ing learning, our approach enables large-scale training of 1D convolutional neural networks while mitigating the issue of label noise in- herent in distant supervision. Our approach achieves state-of-the-art results on minimally supervised sentential relation extraction, out- performing a number of baselines, including a competitive approach that uses the attention layer of a purely neural model.1 1 Introduction Recent years have seen significant progress on tasks such as object detection, automatic speech recognition and machine translation. These per- formance advances are largely driven by the appli- cation of neural network methods on large, high- quality datasets. In contrast, traditional datasets for relation extraction are based on expensive and time-consuming human annotation (Doddington et al., 2004) and are therefore relatively small. Distant supervision (Mintz et al., 2009), a tech- nique which uses existing knowledge bases such as Freebase or Wikipedia as a source of weak su- pervision, enables learning from large quantities of unlabeled text and is a promising approach for scaling up. Recent work has shown promising re- sults from large-scale training of neural networks for relation extraction (Toutanova et al., 2015; Zeng et al., 2015). There are, however, significant challenges due to the inherent noise in distant supervision. For 1 Our code and data are publicly available on Github: https://github.com/bflashcp3f/PCNN-NMAR example, Riedel et al. (2010) showed that, when learning using distant supervision from a knowl- edge base, the portion of mis-labeled examples can vary from 13% to 31%. To address this issue, another line of work has explored structured learn- ing methods that introduce latent variables. An ex- ample is MultiR (Hoffmann et al., 2011), which is based on a joint model of relations between enti- ties in a knowledge base and those mentioned in text. This structured learning approach has a num- ber of advantages; for example, by integrating in- ference into the learning procedure it has the po- tential to overcome the challenge of missing facts by ignoring the knowledge base when mention- level classifiers have high confidence (Ritter et al., 2013; Xu et al., 2013). Prior work on structured learning from minimal supervision has leveraged sparse feature representations, however, and has therefore not benefited from learned representa- tions, which have recently achieved state-of-the- art results on a broad range of NLP tasks. In this paper, we present an approach that com- bines the benefits of structured and neural methods for minimally supervised relation extraction. Our proposed model learns sentence representations that are computed by a 1D convolutional neural network …

Content truncated to 500 words