Latent Topic Refinement based on Distance Metric Learning and Semantics-assisted Non-negative Matrix Factorization

Latent Topic Refinement based on Distance Metric Learning and Semantics-assisted Non-negative Matrix Factorization Tran-Binh Dang, Ha-Thanh Nguyen, Le-Minh Nguyen Japan Advanced Institute of Science and Technology Nomi, Ishikawa, Japan {binhdang, nguyenhathanh, nguyenml}@jaist.ac.jp Abstract SeaNMF stands for the Semantics-assisted non-negative factorization. This approach is the current state-of-the-art method for topic modeling. In this study, we propose a new method (i.e., DML-SeaNMF) for improving the latent topic by utilizing the distance metric learning. The main idea is to iteratively learn the appropriate term-document and term-term relations based on extracted topics in the pre- vious step. Our experiments show that the DML-SeaNMF outperforms the SeaNMF in evaluating based on the topic coherence and topic-based document classification accuracy on several datasets. 1 Introduction With the development of the social network, the textual data repository is being enriched by a huge amount of informative posts, comments, and ques- tions from the internet, by which, we can extract latent information and knowledge by using various text mining methods. Among them, topic model- ing is a well-known problem. For short text data (posts, comments, and questions), there are some methods like biterm topic model (BTM)(Yan et al., 2013), LeadLDA (Li et al., 2016), etc. These meth- ods used topic modeling variants to reduce the ef- fects of sparsity issues on topic modeling. Miao (2017) proposed an approach that used deep neural network architecture for topic modeling. In another way, Non-negative matrix factorization(NMF) is a solution for topic modeling. Choo (2013) success- fully applied this approach. Yan (2013) used fac- torize a symmetric term correlation matrix for topic model. In WWW 2018, the SeaNMF (Tian Shi et al., 2018) was proposed to learn topics from short texts. The model combines document-word relation and word-context relation as inputs. SeaNMF out- performs state-of-the-art methods for topic model- ing such as LDA (Blei et al., 2003), NMF, PTM (Zuo et al., 2016), and GPUDMM (Li et al., 2016). In this paper, we propose a novel method that in- corporates the distance metric learning(DML) for refining latent topics extracted by the SeaNMF, which is called the DML-SeaNMF. A proper topic is a cluster of words that share a common semantic. We suppose that in the top- ics extracted by the SeaNMF, there exists a sub- set of proper topics. Hence, we aim to refine non- proper topics by changing the input matrices of the SeaNMF that is based on the proper ones. To this end, we consider the most likely topic of each word w as the “soft label” of such a word. Besides, each row in the term-document matrix or the term-term matrix (semantic matrix) is a vector representation of each word, denoted by −→v w. Hence, we aim to learn the new representation of word −→ v′ w based on −→v w, which satisfies that vectors representing words in the same topic are close in the vector space (they have the small Euclidean distance). That means the distance deuc(−→ v′ wi, −→ v′ wj) is small if wi and wj …

Content truncated to 500 words