Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1366–1371, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction Jason Weston Google 111 8th avenue New York, NY, USA jweston@google.com Antoine Bordes Heudiasyc UT de Compiègne & CNRS Compiègne, France bordesan@utc.fr Oksana Yakhnenko Google 111 8th avenue New York, NY, USA oksana@google.com Nicolas Usunier Heudiasyc UT de Compiègne & CNRS Compiègne, France nusunier@utc.fr Abstract This paper proposes a novel approach for rela- tion extraction from free text which is trained to jointly use information from the text and from existing knowledge. Our model is based on scoring functions that operate by learning low-dimensional embeddings of words, enti- ties and relationships from a knowledge base. We empirically show on New York Times ar- ticles aligned with Freebase relations that our approach is able to efficiently use the extra in- formation provided by a large subset of Free- base data (4M entities, 23k relationships) to improve over methods that rely on text fea- tures alone. 1 Introduction Information extraction (IE) aims at generating struc- tured data from free text in order to populate Knowl- edge Bases (KBs). Hence, one is given an incom- plete KB composed of a set of triples of the form (h, r, t); h is the left-hand side entity (or head), t the right-hand side entity (or tail) and r the relation- ship linking them. An example from the Freebase KB1 is (/m/2d3rf ,<director-of>, /m/3/324), where /m/2d3rf refers to the director “Alfred Hitchcock" and /m/3/324 to the movie “The Birds". This paper focuses on the problem of learning to perform relation extraction (RE) under weak super- vision from a KB. RE is sub-task of IE that consid- ers that entities have already been detected by a dif- ferent process, such as a named-entity recognizer. RE then aims at assigning to a relation mention m 1www.freebase.com (i.e. a sequence of text which states that some rela- tion is true) the corresponding relationship from the KB, given a pair of extracted entities (h, t) as con- text. For example, given the triple (/m/2d3rf ,“wrote and directed", /m/3/324), a system should predict <director-of>. The task is said to be weakly super- vised because for each pair of entities (h, t) detected in the text, all relation mentions m associated with them are labeled with all the relationships connect- ing h and t in the KB, whether they are actually ex- pressed by m or not. Our key contribution is a novel model that em- ploys not only weakly labeled text mention data, as most approaches do, but also leverages triples from the known KB. The model thus learns the plausi- bility of new (h, r, t) triples by generalizing from the KB, even though this triple is not present. A ranking-based embedding framework is used to train our model. Thereby, relation mentions, entities and relationships are all embedded into a common low- dimensional vector …

Content truncated to 500 words