Incorporating Global Contexts into Sentence Embedding for Relational Extraction at the Paragraph Level with Distant Supervision

Incorporating Global Contexts into Sentence Embedding for Relational Extraction at the Paragraph Level with Distant Supervision Eun-kyung Kim, Key-Sun Choi Semantic Web Research Center, School of Computing, KAIST 291 Daehak-ro, Yuseong-gu, Daejeon, South Korea {kekeeo, kschoi}@kaist.ac.kr Abstract The increased demand for structured knowledge has created considerable interest in relation extraction (RE) from large collections of documents. In particular, distant supervision can be used for RE without manual annotation costs. Nevertheless, this paradigm only extracts relations from individual sentences that contain two target entities. This paper explores the incorporation of global contexts derived from paragraph-into-sentence embedding as a means of compensating for the shortage of training data in distantly supervised RE. Experiments on RE from Korean Wikipedia show that the presented approach can learn an exact RE from sentences (including grammatically incoherent sentences) without syntactic parsing. Keywords: Relation Extraction, Sentence Embedding, Pro-drop Languages 1. Introduction As the demand for structured knowledge has increased, considerable interest has emerged in relation extraction (RE) from large collections of documents written in natu- ral language. In particular, with “distant supervision” (DS) (Mintz et al., 2009; Hoffmann et al., 2011; Riedel et al., 2013), it is possible to extract the relationships between pairs of entities without human manual annotation using a knowledge base (KB); this heuristically aligns entities in texts to a given KB and then uses this alignment to train an RE system. Although the DS strategy is a more effective method of au- tomatically labeling training data than directly supervised labeling, DS-based approaches can extract only relations that are limited to a “single complete sentence” that con- tains two target entities. This makes it difficult to obtain both the subjects and object entities that participate in the KB in a single sentence, particularly in null subject (or object) languages such as Korean, Japanese, Arabic, and Swedish, that can leave the subject of a sentence unex- pressed, unlike English which allows neither. It is also difficult to utilize DS-based approaches for English data when sentences have an informal, grammatically incoher- ent style, such as the style popularly used on Twitter, in dis- charge summaries of clinical texts (Marsh and Sager, 1982), or in a text shortened to bulleted lists in a Wikipedia article. This point can be illustrated by considering the examples in Figure 1. S1 contains a subject, object, and predicate, whereas the subject is omitted in S2 because it is obvious in adjacent sentences in Korean, resulting in differences be- tween the same sentence written in Korean and in English. Therein, we know S2 is obviously a positive example for tuple founderOf(Steve Jobs, Apple Inc.), but we cannot label the training instance S2 according to the traditional paradigm of an existing DS-based approach. We propose a novel approach that performs RE across sen- tences, at the paragraph-level, and does not require labeled data. The proposed method builds upon sentence embed- ding with global context constraints by spanning multiple S1: 스티브잡스는 미국의 기업인 이었다. seutibeujabseuneun migug-ui gieob-in ieossda. Steve Jobs-N in United States a …

Content truncated to 500 words