Global Textual Relation Embedding for Relational Understanding

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1324–1330 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1324 Global Textual Relation Embedding for Relational Understanding Zhiyu Chen1, Hanwen Zha1, Honglei Liu1, Wenhu Chen1, Xifeng Yan1, and Yu Su2 1University of California, Santa Barbara, CA, USA 2The Ohio State University, OH, USA {zhiyuchen, hwzha, honglei, wenhuchen, xyan}@cs.ucsb.edu, su.809@osu.edu Abstract Pre-trained embeddings such as word embed- dings and sentence embeddings are fundamen- tal tools facilitating a wide range of down- stream NLP tasks. In this work, we investigate how to learn a general-purpose embedding of textual relations, defined as the shortest de- pendency path between entities. Textual rela- tion embedding provides a level of knowledge between word/phrase level and sentence level, and we show that it can facilitate downstream tasks requiring relational understanding of the text. To learn such an embedding, we cre- ate the largest distant supervision dataset by linking the entire English ClueWeb09 corpus to Freebase. We use global co-occurrence statistics between textual and knowledge base relations as the supervision signal to train the embedding. Evaluation on two relational understanding tasks demonstrates the useful- ness of the learned textual relation embed- ding. The data and code can be found at https://github.com/czyssrs/GloREPlus 1 Introduction Pre-trained embeddings such as word embeddings (Mikolov et al., 2013; Pennington et al., 2014; Peters et al., 2018; Devlin et al., 2018) and sen- tence embeddings (Le and Mikolov, 2014; Kiros et al., 2015) have become fundamental NLP tools. Learned with large-scale (e.g., up to 800 billion to- kens (Pennington et al., 2014)) open-domain cor- pora, such embeddings serve as a good prior for a wide range of downstream tasks by endowing task-specific models with general lexical, syntac- tic, and semantic knowledge. Inspecting the spectrum of granularity, a repre- sentation between lexical (and phrasal) level and sentence level is missing. Many tasks require rela- tional understanding of the entities mentioned in the text, e.g., relation extraction and knowledge base completion. Textual relation (Bunescu and Mooney, 2005), defined as the shortest path be- tween two entities in the dependency parse tree of a sentence, has been widely shown to be the main bearer of relational information in text and proved effective in relation extraction tasks (Xu et al., 2015; Su et al., 2018). If we can learn a general-purpose embedding for textual relations, it may facilitate many downstream relational un- derstanding tasks by providing general relational knowledge. Similar to language modeling for learning general-purpose word embeddings, distant super- vision (Mintz et al., 2009) is a promising way to acquire supervision, at no cost, for training general-purpose embedding of textual relations. Recently Su et al. (2018) propose to lever- age global co-occurrence statistics of textual and KB relations to learn embeddings of textual re- lations, and show that it can effectively combat the wrong labeling problem of distant supervision (see Figure 1 for example). While their method, named GloRE, achieves the state-of-the-art per- formance on the popular New York …

Content truncated to 500 words