Robust Representation Learning with Reliable Pseudo-labels Generation via Self-Adaptive Optimal Transport for Short Text Clustering

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 10493–10507 July 9-14, 2023 ©2023 Association for Computational Linguistics Robust Representation Learning with Reliable Pseudo-labels Generation via Self-Adaptive Optimal Transport for Short Text Clustering Xiaolin Zheng, Mengling Hu, Weiming Liu, Chaochao Chen∗, and Xinting Liao Zhejiang University, China {xlzheng,humengling,21831010,zjuccc,xintingliao}@zju.edu.cn Abstract Short text clustering is challenging since it takes imbalanced and noisy data as inputs. Ex- isting approaches cannot solve this problem well, since (1) they are prone to obtain de- generate solutions especially on heavy imbal- anced datasets, and (2) they are vulnerable to noises. To tackle the above issues, we propose a Robust Short Text Clustering (RSTC) model to improve robustness against imbalanced and noisy data. RSTC includes two modules, i.e., pseudo-label generation module and robust representation learning module. The former generates pseudo-labels to provide supervision for the later, which contributes to more robust representations and correctly separated clus- ters. To provide robustness against the imbal- ance in data, we propose self-adaptive optimal transport in the pseudo-label generation mod- ule. To improve robustness against the noise in data, we further introduce both class-wise and instance-wise contrastive learning in the robust representation learning module. Our empirical studies on eight short text clustering datasets demonstrate that RSTC significantly outper- forms the state-of-the-art models. The code is available at: https://github.com/hmllmh/RSTC. 1 Introduction Text clustering, one of the most fundamental tasks in text mining, aims to group text instances into clusters in an unsupervised manner. It has been proven to be beneficial in many applications, such as, recommendation system (Liu et al., 2021, 2022a,b), opinion mining (Stieglitz et al., 2018), stance detection (Li et al., 2022), etc. With the advent of digital era, more and more people enjoy sharing and discovering various of contents on the web, where short text is an import form of informa- tion carrier. Therefore, it is helpful to utilize short text clustering for mining valuable insights on the web. ∗Corresponding author. However, short text clustering is not a trivial task. On the one hand, short text has many cate- gories and the category distributions are diversify- ing, where the heavy imbalanced data is common. The heavy imbalanced data is prone to lead to de- generate solutions where the tail clusters (i.e., the clusters with a small proportion of instances) disap- pear. Specifically, the recent deep joint clustering methods for short text clustering, (Hadifar et al., 2019) and (Zhang et al., 2021), adopt the clustering objective proposed in (Xie et al., 2016), which may obtain a trivial solution where all the text instances fall into the same cluster (Yang et al., 2017; Ji et al., 2019). (Zhang et al., 2021) introduces instance- wise contrastive learning to train discriminative representations, which avoids the trivial solution to some extent. However, (Zhang et al., 2021) still tends to generate degenerate solutions, especially on the heavy imbalanced datasets. On the other hand, short text is typically charac- terized by noises, which may lead to meaningless or …

Content truncated to 500 words