A Relational Memory-based Embedding Model for Triple Classification and Search Personalization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3429–3435 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3429 A Relational Memory-based Embedding Model for Triple Classification and Search Personalization Dai Quoc Nguyen1, Tu Dinh Nguyen2, Dinh Phung1 1Monash University, Australia 2Trusting Social 1{dai.nguyen,dinh.phung}@monash.edu 2tu@trustingsocial.com Abstract Knowledge graph embedding methods often suffer from a limitation of memorizing valid triples to predict new ones for triple classi- fication and search personalization problems. To this end, we introduce a novel embed- ding model, named R-MeN, that explores a relational memory network to encode poten- tial dependencies in relationship triples. R- MeN considers each triple as a sequence of 3 input vectors that recurrently interact with a memory using a transformer self-attention mechanism. Thus R-MeN encodes new in- formation from interactions between the mem- ory and each input vector to return a corre- sponding vector. Consequently, R-MeN feeds these 3 returned vectors to a convolutional neu- ral network-based decoder to produce a scalar score for the triple. Experimental results show that our proposed R-MeN obtains state-of-the- art results on SEARCH17 for the search per- sonalization task, and on WN11 and FB13 for the triple classification task. 1 Introduction Knowledge graphs (KGs) – representing the gen- uine relationships among entities in the form of triples (subject, relation, object) denoted as (s, r, o) – are often insufficient for knowledge presentation due to the lack of many valid triples (West et al., 2014). Therefore, research work has been focusing on inferring whether a new triple missed in KGs is likely valid or not (Bordes et al., 2011, 2013; Socher et al., 2013). As summarized in (Nickel et al., 2016; Nguyen, 2017), KG embedding mod- els aim to compute a score for each triple, such that valid triples have higher scores than invalid ones. Early embedding models such as TransE (Bordes et al., 2013), TransH (Wang et al., 2014), TransR (Lin et al., 2015), TransD (Ji et al., 2015), DIST- MULT (Yang et al., 2015) and ComplEx (Trouil- lon et al., 2016) often employ simple linear oper- ators such as addition, subtraction and multiplica- tion. Recent embedding models such as ConvE (Dettmers et al., 2018) and CapsE (Nguyen et al., 2019b) successfully apply deep neural networks to score the triples. Existing embedding models are showing promis- ing performances mainly for knowledge graph com- pletion, where the goal is to infer a missing entity given a relation and another entity. But in real appli- cations, less mentioned, such as triple classification (Socher et al., 2013) that aims to predict whether a given triple is valid, and search personalization (Vu et al., 2017) that aims to re-rank the relevant documents returned by a user-oriented search sys- tem given a query, these models do not effectively capture potential dependencies among entities and relations from existing triples to predict new triples. To this end, we leverage the relational mem- ory network (Santoro et al., 2018) to propose R- MeN to infer a valid fact …

Content truncated to 500 words