Diversity-Aware Batch Active Learning for Dependency Parsing

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 11–26 June 6–11, 2021. ©2021 Association for Computational Linguistics 11 Distantly Supervised Relation Extraction with Sentence Reconstruction and Knowledge Base Priors Fenia Christopoulou1, Makoto Miwa2,3, Sophia Ananiadou1 1National Centre for Text Mining, Department of Computer Science, The University of Manchester, United Kingdom 2Toyota Technological Institute, Nagoya, 468-8511, Japan 3Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology, Japan {efstathia.christopoulou, sophia.ananiadou}@manchester.ac.uk makoto-miwa@toyota-ti.ac.jp Abstract We propose a multi-task, probabilistic approach to facilitate distantly supervised relation extrac- tion by bringing closer the representations of sentences that contain the same Knowledge Base pairs. To achieve this, we bias the la- tent space of sentences via a Variational Au- toencoder (VAE) that is trained jointly with a relation classifier. The latent code guides the pair representations and influences sentence reconstruction. Experimental results on two datasets created via distant supervision indi- cate that multi-task learning results in perfor- mance benefits. Additional exploration of em- ploying Knowledge Base priors into the VAE reveals that the sentence space can be shifted towards that of the Knowledge Base, offering interpretability and further improving results1. 1 Introduction Distant supervision (DS) is a setting where infor- mation from existing, structured knowledge, such as Knowledge Bases (KB), is exploited to automat- ically annotate raw data. For the task of relation extraction, this setting was popularised by Mintz et al. (2009). Sentences containing a pair of interest were annotated as positive instances of a relation, if and only if the pair was found to share this relation in the KB. However, due to the strictness of this assumption, relaxations were proposed, such as the at-least-one assumption introduced by Riedel et al. (2010): Instead of assuming that all sentences in which a known related pair appears express the relationship, we assume that at least one of these sentences (namely a bag of sentences) expresses the relationship. Figure 1 shows example bags for two entity pairs. 1Source code is available at https://github.com/ fenchri/dsre-vae Among other reasons , Apple 's chief executive , Steve Jobs , ... About Apple 's Steve Jobs , who bought out ... /people/person/place_of_birth Mayor Ray Nagin born in New Orleans has already ... C. Ray Nagin , the mayor of New Orleans , ... Relation /business/company/founders Entity 1 Steve Jobs Ray Nagin Entity 2 Apple New Orleans Link Prediction bag 1 bag 2 Figure 1: Example of the bag-level setting in distantly supervised relation extraction and the main idea of our approach. Sentences are adapted from the NYT10 dataset (Riedel et al., 2010). The usefulness of distantly supervised relation extraction (DSRE) is reflected in facilitating au- tomatic data annotation, as well as the usage of such data to train models for KB population (Ji and Grishman, 2011). However, DSRE suffers from noisy instances, long-tail relations and unbalanced bag sizes. Typical noise reduction methods have focused on using attention (Lin et al., 2016; Ye and Ling, 2019) or reinforcement …

Content truncated to 500 words