Self-Attention Enhanced CNNs and Collaborative Curriculum Learning for Distantly Supervised Relation Extraction

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 389–398, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 389 Self-Attention Enhanced CNNs and Collaborative Curriculum Learning for Distantly Supervised Relation Extraction Yuyun Huang† Jinhua Du‡§ †University College Dublin, Ireland ‡Investments AI, AIG (American International Group, Inc.) §ADAPT Centre, School of Computing, Dublin City University, Ireland {yuyun.huang}@ucd.ie {jinhua.du}@aig.com Abstract Distance supervision is widely used in relation extraction tasks, particularly when large-scale manual annotations are virtually impossible to conduct. Although Distantly Supervised Re- lation Extraction (DSRE) benefits from auto- matic labelling, it suffers from serious mis- labelling issues, i.e. some or all of the in- stances for an entity pair (head and tail enti- ties) do not express the labelled relation. In this paper, we propose a novel model that employs a collaborative curriculum learning framework to reduce the effects of mislabelled data. Specifically, we firstly propose an in- ternal self-attention mechanism between the convolution operations in convolutional neu- ral networks (CNNs) to learn a better sentence representation from the noisy inputs. Then we define two sentence selection models as two relation extractors in order to collaboratively learn and regularise each other under a curricu- lum scheme to alleviate noisy effects, where the curriculum could be constructed by con- flicts or small loss. Finally, experiments are conducted on a widely-used public dataset and the results indicate that the proposed model significantly outperforms baselines including the state-of-the-art in terms of P@N and PR curve metrics, thus evidencing its capability of reducing noisy effects for DSRE. 1 Introduction Relation Extraction (RE) is vital for NLP tasks such as information extraction, question answer- ing and knowledge base completion. RE aims to identify the relationship between an entity pair (e1, e2) in a sentence. For example, in the sen- tence “[Bill Gatese1] is the principal founder of [Microsofte2]”, the relation extractor decodes the relation of founder for the entity pair Bill Gates (the person) and Microsoft (the company). Recent supervised relation extraction research can be roughly categorised into two areas: fully supervised and distantly supervised relation ex- traction. Fully supervised relation extraction mainly depends on manually annotated training dataset (Zeng et al., 2014; dos Santos et al., 2015). Ordinarily, human annotation on large- scale datasets is costly and often practicably im- possible. Distance supervision addresses this problem by using an existing knowledge base (e.g. DBpedia) to automatically annotating large-scale datasets, thus reducing the burden of manual anno- tation (Mintz et al., 2009; Hoffmann et al., 2011; Surdeanu et al., 2012). However, distance super- vision often suffers from mislabelling problems. Figure 1 illustrates this incorrect labelling is- sue, which shows that not all sentences in a bag with the same entity pair express the labelled rela- tion of person/company/founder. The worst case is that all sentences in a bag are mislabelled. Thus, one primary challenge in DSRE is to minimize the noisy labelling effects, which in turn, would let model …

Content truncated to 500 words