Unsupervised Few-Bits Semantic Hashing with Implicit Topics Modeling

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2566–2575 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2566 Unsupervised Few-Bits Semantic Hashing with Implicit Topics Modeling Fanghua Ye, Jarana Manotumruksa, Emine Yilmaz University College London, UK {fanghua.ye.19, j.manotumruksa, emine.yilmaz}@ucl.ac.uk Abstract Semantic hashing is a powerful paradigm for representing texts as compact binary hash codes. The explosion of short text data has spurred the demand of few-bits hashing. How- ever, the performance of existing semantic hashing methods cannot be guaranteed when applied to few-bits hashing because of severe information loss. In this paper, we present a simple but effective unsupervised neural gener- ative semantic hashing method with a focus on few-bits hashing. Our model is built upon vari- ational autoencoder and represents each hash bit as a Bernoulli variable, which allows the model to be end-to-end trainable. To address the issue of information loss, we introduce a set of auxiliary implicit topic vectors. With the aid of these topic vectors, the generated hash codes are not only low-dimensional rep- resentations of the original texts but also cap- ture their implicit topics. We conduct compre- hensive experiments on four datasets. The re- sults demonstrate that our approach achieves significant improvements over state-of-the-art semantic hashing methods in few-bits hashing. 1 Introduction Semantic hashing (Salakhutdinov and Hinton, 2009) is an attractive strategy for fast similarity search, which aims to find the most relevant texts for a given query (Wang et al., 2017). The basic idea of semantic hashing is to embed the semantics of texts into a low-dimensional binary vector space, while preserving text similarity. The embedded rep- resentations are called hash codes, based on which the calculation of text similarity can be efficiently completed by computing the Hamming distance using XOR operation (Zhang et al., 2010). While considerable research efforts have been devoted to semantic hashing (Wang et al., 2013; Xu et al., 2015; Chaidaroon and Fang, 2017; Shen et al., 2018; Dong et al., 2019; Hansen et al., 2019; 0 0 1 1 0 𝑡𝑡1𝑡𝑡2𝑡𝑡3 𝑡𝑡𝑙𝑙 𝑡𝑡1𝑡𝑡3 𝑑𝑑 ̅𝑧𝑧 𝑧𝑧 𝑔𝑔(𝒯𝒯𝑧𝑧) ̂𝑑𝑑 𝒯𝒯𝑧𝑧 ⋯ Topic vector selection 0.9 0.1 0.8 0.2 0.1 𝜎𝜎(𝑟𝑟(𝑑𝑑)) Inference network Generative network Figure 1: The architecture of our approach WISH. The inference network maps text d to hash code z and the generative network reconstructs d based on the selected topic vectors Tz. z captures the implicit topics of d. Dadaneh et al., 2020), none of them have paid at- tention to few-bits hashing. Their performance also cannot be guaranteed when directly applied to few-bits hashing due to severe information loss. However, compactness is a crucial factor in learn- ing to hash (Wang et al., 2015). It is important to keep hash codes as short as possible. For a text collection with c topics, the ideal length of hash codes is just ⌈log2 (c)⌉ (Liu et al., 2019). In addi- tion, with the explosive growth of social media and e-commerce, more and more short text data (e.g., tweets and online reviews) are generated …

Content truncated to 500 words