Rationalizing Medical Relation Prediction from Corpus-level Statistics

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 777–788 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 777 Generative Semantic Hashing Enhanced via Boltzmann Machines Lin Zheng1, Qinliang Su1,2∗, Dinghan Shen3, Changyou Chen4 1School of Data and Computer Science, Sun Yat-sen University 2Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China 3Microsoft Dynamics 365 AI 4CSE Department, SUNY at Buffalo zhenglin6@mail2.sysu.edu.cn, suqliang@mail.sysu.edu.cn dishen@microsoft.com, changyou@buffalo.edu Abstract Generative semantic hashing is a promis- ing technique for large-scale information re- trieval thanks to its fast retrieval speed and small memory footprint. For the tractability of training, existing generative-hashing meth- ods mostly assume a factorized form for the posterior distribution, enforcing independence among the bits of hash codes. From the per- spectives of both model representation and code space size, independence is always not the best assumption. In this paper, to intro- duce correlations among the bits of hash codes, we propose to employ the distribution of Boltz- mann machine as the variational posterior. To address the intractability issue of training, we first develop an approximate method to repa- rameterize the distribution of a Boltzmann ma- chine by augmenting it as a hierarchical con- catenation of a Gaussian-like distribution and a Bernoulli distribution. Based on that, an asymptotically-exact lower bound is further derived for the evidence lower bound (ELBO). With these novel techniques, the entire model can be optimized efficiently. Extensive ex- perimental results demonstrate that by effec- tively modeling correlations among different bits within a hash code, our model can achieve significant performance gains. 1 Introduction Similarity search, also known as nearest-neighbor search, aims to find items that are similar to a query from a large dataset. It plays an impor- tant role in modern information retrieval systems and has been used in various applications, rang- ing from plagiarism analysis (Stein et al., 2007) to content-based multimedia retrieval (Lew et al., 2006), etc. However, looking for nearest neighbors in the Euclidean space is often computationally ∗Corresponding author. prohibitive for large-scale datasets (calculating co- sine similarity with high-dimensional vectors is computationally-expensive). Semantic hashing cir- cumvents this problem by representing semanti- cally similar documents with compact and binary codes. Accordingly, similar documents can be re- trieved by evaluating the hamming distances of their hash codes much more efficiently. To obtain similarity-preserving hash codes, ex- tensive efforts have been made to learn hash func- tions that can preserve the similarity information of original documents in the binary embedding space (Shen et al., 2015; Liu et al., 2016). Existing meth- ods often require the availability of label informa- tion, which is often expensive to obtain in practice. To avoid the use of labels, generative semantic hash- ing methods have been developed. Specifically, the variational autoencoder (VAE) is first employed for semantic hashing in (Chaidaroon and Fang, 2017), and their model is termed VDSH. As a two-step process, the continuous document representations obtained from VAE are directly converted into bi- nary hash codes. To resolve the two-step training problem, …

Content truncated to 500 words