A Systematic Investigation of KB-Text Embedding Alignment at Scale

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1764–1774 August 1–6, 2021. ©2021 Association for Computational Linguistics 1764 A Systematic Investigation of KB-Text Embedding Alignment at Scale Vardaan Pahuja1, Yu Gu1, Wenhu Chen2, Mehdi Bahrami3, Lei Liu3, Wei-Peng Chen3, Yu Su1 1The Ohio State University 2University of California, Santa Barbara 3Fujitsu Laboratories of America {pahuja.9, gu.826, su.809}@osu.edu, wenhuchen@cs.ucsb.edu {mbahrami, lliu, wchen}@fujitsu.com Abstract Knowledge bases (KBs) and text often con- tain complementary knowledge: KBs store structured knowledge that can support long- range reasoning, while text stores more com- prehensive and timely knowledge in an un- structured way. Separately embedding the in- dividual knowledge sources into vector spaces has demonstrated tremendous successes in en- coding the respective knowledge, but how to jointly embed and reason with both knowledge sources to fully leverage the complementary information is still largely an open problem. We conduct a large-scale, systematic investiga- tion of aligning KB and text embeddings for joint reasoning. We set up a novel evaluation framework with two evaluation tasks, few-shot link prediction and analogical reasoning, and evaluate an array of KB-text embedding align- ment methods. We also demonstrate how such alignment can infuse textual information into KB embeddings for more accurate link pre- diction on emerging entities and events, using COVID-19 as a case study.1 1 Introduction Recent years have witnessed a rapid growth of knowledge bases (KBs) such as Freebase (Bol- lacker et al., 2007), DBPedia (Auer et al., 2007), YAGO (Suchanek et al., 2007) and Wikidata (Vrandeˇci´c and Kr¨otzsch, 2014). These KBs store facts about real-world entities (e.g. people, places, and things) in the form of RDF triples, i.e. (sub- ject, predicate, object). Today’s KBs are massive in scale. For instance, Freebase contains over 45 mil- lion entities and 3 billion facts involving a large va- riety of relations. Such large-scale multi-relational knowledge provides a great potential for improv- ing a wide range of tasks, from information re- trieval (Castells et al., 2007; Shen et al., 2015), 1Code and data are available at https://github. com/dki-lab/joint-kb-text-embedding. Article: United Nations Secretariat Headquartered in New York, the Secretariat functions through duty stations in Addis Ababa, Bangkok, Beirut, Geneva, Nairobi, Santiago and Vienna, in addition to offices all over the world United Nations Secretariat United Nations System United Nations New York part of parent organization headquarters location Figure 1: KBs and text are complementary and embed- ding alignment could help injecting information from one source to the other and vice versa. Dashed line is missing link in the KB. question answering (Yao and Van Durme, 2014; Yu et al., 2017) to biological data mining (Zheng et al., 2020b). KB embedding models (Bordes et al., 2013; Dong et al., 2014; Lin et al., 2015) embed enti- ties and relations into vector space(s) such that the embeddings capture the symbolic knowledge present in the KB. Similarly, word embedding mod- els (Mikolov et al., 2013b; Pennington et al., 2014) learn continuous vector representations …

Content truncated to 500 words