Document Hashing with Mixture-Prior Generative Models

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5226–5235, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5226 Document Hashing with Mixture-Prior Generative Models Wei Dong1, Qinliang Su1, 2∗, Dinghan Shen3, Changyou Chen4 1 School of Data and Computer Science, Sun Yat-sen University 2 Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China 3 ECE Department, Duke University 4 CSE Department, SUNY at Buffalo dongw23@mail2.sysu.edu.cn, suqliang@mail.sysu.edu.cn dinghan.shen@duke.edu, changyou@buffalo.edu Abstract Hashing is promising for large-scale informa- tion retrieval tasks thanks to the efficiency of distance evaluation between binary codes. Generative hashing is often used to generate hashing codes in an unsupervised way. How- ever, existing generative hashing methods on- ly considered the use of simple priors, like Gaussian and Bernoulli priors, which limit- s these methods to further improve their per- formance. In this paper, two mixture-prior generative models are proposed, under the ob- jective to produce high-quality hashing codes for documents. Specifically, a Gaussian mix- ture prior is first imposed onto the variation- al auto-encoder (VAE), followed by a separate step to cast the continuous latent representa- tion of VAE into binary code. To avoid the performance loss caused by the separate cast- ing, a model using a Bernoulli mixture pri- or is further developed, in which an end-to- end training is admitted by resorting to the straight-through (ST) discrete gradient estima- tor. Experimental results on several bench- mark datasets demonstrate that the proposed methods, especially the one using Bernoulli mixture priors, consistently outperform exist- ing ones by a substantial margin. 1 Introduction Similarity search aims to find items that look most similar to the query one from a huge amount of data (Wang et al., 2018), and are found in exten- sive applications like plagiarism analysis (Stein et al., 2007), collaborative filtering (Koren, 2008; Wang et al., 2016), content-based multimedia re- trieval (Lew et al., 2006), web services (Dong et al., 2004) etc. Semantic hashing is an effective way to accelerate the searching process by rep- resenting every document with a compact binary code. In this way, one only needs to evaluate the ∗Corresponding author. hamming distance between binary codes, which is much cheaper than the Euclidean distance calcu- lation in the original feature space. Existing hashing methods can be roughly divid- ed into data-independent and data-dependent cat- egories. Data-independent methods employ ran- dom projections to construct hash functions with- out any consideration on data characteristics, like the locality sensitive hashing (LSH) algorithm (Datar et al., 2004). On the contrary, data depen- dent hashing seeks to learn a hash function from the given training data in a supervised or an un- supervised way. In the supervised case, a deter- ministic function which maps the data to a bina- ry representation is trained by using the provided supervised information (e.g. labels) (Liu et al., 2012; Shen et al., 2015; Liu et al., 2016). How- ever, the supervised information is often …

Content truncated to 500 words