Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2238–2249 August 1–6, 2021. ©2021 Association for Computational Linguistics 2238 Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval Zijing Ou1, Qinliang Su1∗, Jianxing Yu2, Bang Liu3, Jingwen Wang4, Ruihui Zhao4, Changyou Chen5 and Yefeng Zheng4 1School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China, 2School of Artificial Intelligence, Sun Yat-sen University, Guangdong, China, 3RALI & Mila, Universit´e de Montr´eal, 4Tencent Jarvis Lab, 5CSE Department, SUNY at Buffalo ouzj@mail2.sysu.edu.cn, {suqliang, yujx26}@mail.sysu.edu.cn, bang.liu@umontreal.ca, {v wjwowang, zacharyzhao, yefengzheng}@tencent.com, changyou@buffalo.edu Abstract With the need of fast retrieval speed and small memory footprint, document hashing has been playing a crucial role in large-scale informa- tion retrieval. To generate high-quality hash- ing code, both semantics and neighborhood information are crucial. However, most ex- isting methods leverage only one of them or simply combine them via some intuitive cri- teria, lacking a theoretical principle to guide the integration process. In this paper, we encode the neighborhood information with a graph-induced Gaussian distribution, and pro- pose to integrate the two types of information with a graph-driven generative model. To deal with the complicated correlations among doc- uments, we further propose a tree-structured approximation method for learning. Under the approximation, we prove that the training ob- jective can be decomposed into terms involv- ing only singleton or pairwise documents, en- abling the model to be trained as efficiently as uncorrelated ones. Extensive experimental re- sults on three benchmark datasets show that our method achieves superior performance over state-of-the-art methods, demonstrating the effectiveness of the proposed model for si- multaneously preserving semantic and neigh- borhood information.1 1 Introduction Similarity search plays a pivotal role in a variety of tasks, such as image retrieval (Jing and Baluja, 2008; Zhang et al., 2018), plagiarism detection (Stein et al., 2007) and recommendation systems (Koren, 2008). If the search is carried out in the original continuous feature space directly, the re- quirements of computation and storage would be ∗Corresponding author. Qinliang Su is also affiliated with (i) Guangdong Key Lab. of Big Data Analysis and Processing, Guangzhou, China, and (ii) Key Lab. of Machine Intelligence and Advanced Computing, Ministry of Education, China. 1Our code is available at https://github.com/J-zin/SNUH. The MindSpore code will also be released soon. extremely high, especially for large-scale applica- tions. Semantic hashing (Salakhutdinov and Hin- ton, 2009b) sidesteps this problem by learning a compact binary code for every item such that simi- lar items can be efficiently found according to the Hamming distance of binary codes. Unsupervised semantic hashing aims to learn for each item a binary code that can preserve the semantic similarity information of original items, without the supervision of any labels. Motivated by the success of deep generative models (Salakhutdi- nov and Hinton, 2009a; Kingma and Welling, 2013; Rezende et al., 2014) in unsupervised representa- tion learning, many recent methods approach this problem from the perspective of deep generative …

Content truncated to 500 words