Multilingual Relation Extraction using Compositional Universal Schema

Proceedings of NAACL-HLT 2016, pages 886–896, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Multilingual Relation Extraction using Compositional Universal Schema Patrick Verga, David Belanger, Emma Strubell, Benjamin Roth & Andrew McCallum College of Information and Computer Sciences University of Massachusetts Amherst {pat, belanger, strubell, beroth, mccallum}@cs.umass.edu Abstract Universal schema builds a knowledge base (KB) of entities and relations by jointly embedding all re- lation types from input KBs as well as textual pat- terns observed in raw text. In most previous appli- cations of universal schema, each textual pattern is represented as a single embedding, preventing gen- eralization to unseen patterns. Recent work employs a neural network to capture patterns’ compositional semantics, providing generalization to all possible input text. In response, this paper introduces sig- nificant further improvements to the coverage and flexibility of universal schema relation extraction: predictions for entities unseen in training and mul- tilingual transfer learning to domains with no an- notation. We evaluate our model through extensive experiments on the English and Spanish TAC KBP benchmark, outperforming the top system from TAC 2013 slot-filling using no handwritten patterns or ad- ditional annotation. We also consider a multilingual setting in which English training data entities over- lap with the seed KB, but Spanish text does not. Despite having no annotation for Spanish data, we train an accurate predictor, with additional improve- ments obtained by tying word embeddings across languages. Furthermore, we find that multilingual training improves English relation extraction accu- racy. Our approach is thus suited to broad-coverage automated knowledge base construction in a variety of languages and domains. 1 Introduction The goal of automatic knowledge base construction (AKBC) is to build a structured knowledge base (KB) of facts using a noisy corpus of raw text evidence, and perhaps an initial seed KB to be augmented (Carlson et al., 2010; Suchanek et al., 2007; Bollacker et al., 2008). AKBC supports downstream reasoning at a high level about extracted entities and their relations, and thus has broad-reaching applications to a variety of domains. One challenge in AKBC is aligning knowledge from a structured KB with a text corpus in order to perform supervised learning through distant supervision. Univer- sal schema (Riedel et al., 2013) along with its exten- sions (Yao et al., 2013; Gardner et al., 2014; Neelakantan et al., 2015; Rocktaschel et al., 2015), avoids alignment by jointly embedding KB relations, entities, and surface text patterns. This propagates information between KB annotation and corresponding textual evidence. The above applications of universal schema express each text relation as a distinct item to be embedded. This harms its ability to generalize to inputs not precisely seen at training time. Recently, Toutanova et al. (2015) ad- dressed this issue by embedding text patterns using a deep sentence encoder, which captures the compositional se- mantics of textual relations and allows for prediction on inputs never seen before. This paper further expands the coverage abilities of universal schema relation extraction by introducing tech- niques for forming predictions for …

Content truncated to 500 words