Improving Deep Embedded Clustering via Learning Cluster-level Representations

Proceedings of the 29th International Conference on Computational Linguistics, pages 2226–2236 October 12–17, 2022. 2226 Improving Deep Embedded Clustering via Learning Cluster-level Representations Qing Yin21, Zhihua Wang34, Yunya Song5, Yida Xu6, Shuai Niu2, Liang Bai7, Yike Guo2 and Xian Yang1∗ 1 Alliance Manchester Business School, The University of Manchester 2 Department of Computer Science, Hong Kong Baptist University 3 Shanghai Institute for Advanced Study of Zhejiang University 4 Shanghai AI Laboratory 5 Department of Journalism, Hong Kong Baptist University 6 Department of Mathematics, Hong Kong Baptist University 7 School of Computer and Information Technology, Shanxi University Abstract Driven by recent advances in neural networks, various Deep Embedding Clustering (DEC) based short text clustering models are being developed. In these works, latent representa- tion learning and text clustering are performed simultaneously. Although these methods are becoming increasingly popular, they use pure cluster-oriented objectives, which can produce meaningless representations. To alleviate this problem, several improvements have been de- veloped to introduce additional learning objec- tives in the clustering process, such as models based on contrastive learning. However, exist- ing efforts rely heavily on learning meaning- ful representations at the instance level. They have limited focus on learning global repre- sentations, which are necessary to capture the overall data structure at the cluster level. In this paper, we propose a novel DEC model, which we named the deep embedded clustering model with cluster-level representation learning (DEC- CRL) to jointly learn cluster and instance level representations. Here, we extend the embedded topic modelling approach to introduce recon- struction constraints to help learn cluster-level representations. Experimental results on real- world short text datasets demonstrate that our model produces meaningful clusters. 1 Introduction Short Text Clustering has gained increasing atten- tion in many real-world applications, such as event discovery (Atefeh and Khreich, 2015), spam detec- tion (Wu and Liu, 2018), and sentiment analysis (Paltoglou and Thelwall, 2012). Unlike long texts, which can be represented as, for instance, term frequency inverse-document-frequency (TF-IDF) vectors in the clustering task, short texts cannot ∗This is the corresponding author. be encoded in the same manner. This is because the vector representation of short texts is highly sparse, making it difficult to measure the similarity between two sets of short texts (Xu et al., 2017). With this observation, deep clustering methods are being developed to encode raw short texts into la- tent representational space using neural networks and to detect clusters based on the learned repre- sentations. Deep clustering methods generally fall into two categories: the two-stage methods and the deep embedded clustering (DEC) methods. The two-stage methods (Zakaria et al., 2012; Tian et al., 2014; Vincent et al., 2010) assign data samples to different clusters after latent representations are learned and fixed, while the DEC methods (Xie et al., 2016) simultaneously learn latent representa- tions and discover clusters via end-to-end training. Different from the two-stage methods, the DEC methods explicitly define the cluster-oriented loss to jointly map raw data into latent representations and acquire cluster assignments. As discussed in (Jiang et …

Content truncated to 500 words