Extracting Relations between Non-Standard Entities using Distant Supervision and Imitation Learning

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 747–757, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Extracting Relations between Non-Standard Entities using Distant Supervision and Imitation Learning Isabelle Augenstein∗ Andreas Vlachos# Diana Maynard∗ ∗ Department of Computer Science, University of Sheffield i.augenstein@sheffield.ac.uk, d.maynard@sheffield.ac.uk # Computer Science Department, University College London a.vlachos@cs.ucl.ac.uk Abstract Distantly supervised approaches have be- come popular in recent years as they allow training relation extractors without text- bound annotation, using instead known relations from a knowledge base and a large textual corpus from an appropri- ate domain. While state of the art dis- tant supervision approaches use off-the- shelf named entity recognition and clas- sification (NERC) systems to identify re- lation arguments, discrepancies in domain or genre between the data used for NERC training and the intended domain for the relation extractor can lead to low perfor- mance. This is particularly problematic for “non-standard” named entities such as album which would fall into the MISC category. We propose to ameliorate this issue by jointly training the named entity classifier and the relation extractor using imitation learning which reduces struc- tured prediction learning to classification learning. We further experiment with Web features different features and com- pare against using two off-the-shelf su- pervised NERC systems, Stanford NER and FIGER, for named entity classifica- tion. Our experiments show that imita- tion learning improves average precision by 4 points over an one-stage classification model, while removing Web features re- sults in a 6 points reduction. Compared to using FIGER and Stanford NER, average precision is 10 points and 19 points higher with our imitation learning approach. 1 Introduction Factual answers to queries such as “What albums did The Beatles release?” are commonly stored in knowledge bases and can then be accessed by an information retrieval system, a commercial exam- ple for this being Google’s knowledge vault (Xin et al., 2014). In order to keep knowledge bases up to date should new facts emerge, and to quickly adapt to new domains, there is a need for flexi- ble and accurate information extraction (IE) ap- proaches which do not require manual effort to be developed for new domains. A popular approach for creating IE methods to extract such relations is distant supervision (Craven and Kumlien, 1999; Mintz et al., 2009) which is a method for learn- ing relation extractors using relations stored in a knowledge base combined with raw text to auto- matically generate training data. An important first step in distant supervision is to identify named entities (NEs) and their types to determine if a pair of NEs is a suitable candi- date for the relation. As an example, the album relation has a Musical Artist and an Album as ar- guments. Existing works use supervised named entity recognisers and classifiers (NERC) with ei- ther a small set of types such as the Stanford NER system (Manning et al., 2014), or fine-grained NE types (Ling and Weld, 2012; Liu et al., 2014). However, …

Content truncated to 500 words