Effects of Creativity and Cluster Tightness on Short Text Clustering Performance

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 654–665, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Effects of Creativity and Cluster Tightness on Short Text Clustering Performance Catherine Finegan-Dollak1 Reed Coke1 Rui Zhang1 Xiangyi Ye2 Dragomir Radev1 {cfdollak, reedcoke, ryanzh, yexy, radev}@umich.edu 1Department of EECS, University of Michigan, Ann Arbor, MI USA 2Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI USA Abstract Properties of corpora, such as the diver- sity of vocabulary and how tightly related texts cluster together, impact the best way to cluster short texts. We examine several such properties in a variety of corpora and track their effects on various combinations of similarity metrics and clustering algo- rithms. We show that semantic similar- ity metrics outperform traditional n-gram and dependency similarity metrics for k- means clustering of a linguistically cre- ative dataset, but do not help with less creative texts. Yet the choice of simi- larity metric interacts with the choice of clustering method. We find that graph- based clustering methods perform well on tightly clustered data but poorly on loosely clustered data. Semantic similarity met- rics generate loosely clustered output even when applied to a tightly clustered dataset. Thus, the best performing clustering sys- tems could not use semantic metrics. 1 Introduction Corpora of collective discourse—texts generated by multiple authors in response to the same stimulus—have varying properties depending on the stimulus and goals of the authors. For instance, when multiple puzzle-composers write crossword puzzle clues for the same word, they will try to write creative, unique clues to make the puzzle in- teresting and challenging; clues for “star” could be “Paparazzi’s target” or “Sky light.” In contrast, people writing a descriptive caption for a photo- graph can adopt a less creative style. Corpora may also differ on how similar texts within a particular class are to one another, compared to how simi- lar they are to texts from other classes. For ex- ample, entries in a cartoon captioning contest that all relate to the same cartoon may vary widely in subject, while crossword clues for the same word would likely be more tightly clustered. This paper studies how such text properties af- fect the best method of clustering short texts. Choosing how to cluster texts involves two ma- jor decisions: choosing a similarity metric to de- termine which texts are alike, and choosing a clustering method to group those texts. We hy- pothesize that creativity may drive authors to ex- press the same concept in a wide variety of ways, leading to data that can benefit from different similarity metrics than less creative texts. At the same time, we hypothesize that tightly clus- tered datasets—datasets where each text is much more similar to texts in its cluster than to texts from other clusters—can be clustered by power- ful graph-based methods such as Markov Cluster- ing (MCL) and Louvain, which may fail on more loosely clustered data. This paper explores the in- teraction of these effects. …

Content truncated to 500 words