STransE: a novel embedding model of entities and relationships in knowledge bases

Proceedings of NAACL-HLT 2016, pages 460–466, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics STransE: a novel embedding model of entities and relationships in knowledge bases Dat Quoc Nguyen1, Kairit Sirts1, Lizhen Qu2 and Mark Johnson1 1 Department of Computing, Macquarie University, Sydney, Australia dat.nguyen@students.mq.edu.au, {kairit.sirts, mark.johnson}@mq.edu.au 2 NICTA, ACT 2601, Australia lizhen.qu@nicta.com.au Abstract Knowledge bases of real-world facts about entities and their relationships are useful re- sources for a variety of natural language pro- cessing tasks. However, because knowledge bases are typically incomplete, it is useful to be able to perform link prediction, i.e., pre- dict whether a relationship not in the knowl- edge base is likely to be true. This paper com- bines insights from several previous link pre- diction models into a new embedding model STransE that represents each entity as a low- dimensional vector, and each relation by two matrices and a translation vector. STransE is a simple combination of the SE and TransE models, but it obtains better link prediction performance on two benchmark datasets than previous embedding models. Thus, STransE can serve as a new baseline for the more com- plex models in the link prediction task. 1 Introduction Knowledge bases (KBs), such as WordNet (Fell- baum, 1998), YAGO (Suchanek et al., 2007), Free- base (Bollacker et al., 2008) and DBpedia (Lehmann et al., 2015), represent relationships between en- tities as triples (head entity, relation, tail entity). Even very large knowledge bases are still far from complete (Socher et al., 2013; West et al., 2014). Link prediction or knowledge base completion sys- tems (Nickel et al., 2015) predict which triples not in a knowledge base are likely to be true (Taskar et al., 2004; Bordes et al., 2011). A variety of differ- ent kinds of information is potentially useful here, including information extracted from external cor- pora (Riedel et al., 2013; Wang et al., 2014a) and the other relationships that hold between the enti- ties (Angeli and Manning, 2013; Zhao et al., 2015). For example, Toutanova et al. (2015) used informa- tion from the external ClueWeb-12 corpus to signif- icantly enhance performance. While integrating a wide variety of information sources can produce excellent results, there are sev- eral reasons for studying simpler models that di- rectly optimize a score function for the triples in a knowledge base, such as the one presented here. First, additional information sources might not be available, e.g., for knowledge bases for specialized domains. Second, models that don’t exploit external resources are simpler and thus typically much faster to train than the more complex models using addi- tional information. Third, the more complex mod- els that exploit external information are typically extensions of these simpler models, and are often initialized with parameters estimated by such sim- pler models, so improvements to the simpler mod- els should yield corresponding improvements to the more complex models as well. Embedding models for KB completion associate entities and/or relations with dense feature vectors or matrices. Such models obtain state-of-the-art per- formance …

Content truncated to 500 words