Semi-supervised Clustering for Short Text via Deep Representation Learning

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 31–39, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Semi-supervised Clustering for Short Text via Deep Representation Learning Zhiguo Wang and Haitao Mi and Abraham Ittycheriah IBM T.J. Watson Research Center Yorktown Heights, NY, USA {zhigwang, hmi, abei}@us.ibm.com Abstract In this work, we propose a semi- supervised method for short text clus- tering, where we represent texts as dis- tributed vectors with neural networks, and use a small amount of labeled data to specify our intention for clustering. We design a novel objective to combine the representation learning process and the k- means clustering process together, and op- timize the objective with both labeled data and unlabeled data iteratively until conver- gence through three steps: (1) assign each short text to its nearest centroid based on its representation from the current neural networks; (2) re-estimate the cluster cen- troids based on cluster assignments from step (1); (3) update neural networks ac- cording to the objective by keeping cen- troids and cluster assignments fixed. Ex- perimental results on four datasets show that our method works significantly better than several other text clustering methods. 1 Introduction Text clustering is a fundamental problem in text mining and information retrieval. Its task is to group similar texts together such that texts within a cluster are more similar to texts in other clus- ters. Usually, a text is represented as a bag-of- words or term frequency-inverse document fre- quency (TF-IDF) vector, and then the k-means al- gorithm (MacQueen, 1967) is performed to parti- tion a set of texts into homogeneous groups. However, when dealing with short texts, the characteristics of short text and clustering task raise several issues for the conventional unsuper- vised clustering algorithms. First, the number of uniqe words in each short text is small, as a re- (a) What’s the color of apples? (b) When will this apple be ripe? (c) Do you like apples? (d) What’s the color of oranges? (e) When will this orange be ripe? (f) Do you like oranges? Table 1: Examples for short text clustering. sult, the lexcical sparsity issue usually leads to poor clustering quality (Dhillon and Guan, 2003). Second, for a specific short text clustering task, we have prior knowledge or particular intentions before clustering, while fully unsupervised ap- proaches may learn some classes the other way around. Take the sentences in Table 1 for exam- ple, those sentences can be clustered into different partitions based on different intentions: apple {a, b, c} and orange {d, e, f} with a fruit type inten- tion, or what-question {a, d}, when-question {b, e}, and yes/no-question cluster {c, f} with a ques- tion type intension. To address the lexical sparity issue, one direc- tion is to enrich text representations by extracting features and relations from Wikipedia (Banerjee et al., 2007) or an ontology (Fodeh et al., 2011). But this approach requires the annotated knowledge, which is also language dependent. So the other direction, …

Content truncated to 500 words