Distributed Document and Phrase Co-embeddings for Descriptive Clustering

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 991–1001, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Distributed Document and Phrase Co-embeddings for Descriptive Clustering Motoki Sato1, Austin J. Brockmeier2, Georgios Kontonatsios1, Tingting Mu1, John Y. Goulermas2, Jun’ichi Tsujii3,1 and Sophia Ananiadou1 1University of Manchester, National Centre for Text Mining (NaCTeM), Manchester, UK 2Department of Computer Science, University of Liverpool, Liverpool, UK 3Artificial Intelligence Research Center, AIST, Tokyo, Japan sato.motoki.sa7@is.naist.jp, a.j.brockmeier@liverpool.ac.uk sophia.ananiadou@manchester.ac.uk Abstract Descriptive document clustering aims to automatically discover groups of seman- tically related documents and to assign a meaningful label to characterise the content of each cluster. In this paper, we present a descriptive clustering ap- proach that employs a distributed repre- sentation model, namely the paragraph vector model, to capture semantic similar- ities between documents and phrases. The proposed method uses a joint representa- tion of phrases and documents (i.e., a co- embedding) to automatically select a de- scriptive phrase that best represents each document cluster. We evaluate our method by comparing its performance to an ex- isting state-of-the-art descriptive cluster- ing method that also uses co-embedding but relies on a bag-of-words represen- tation. Results obtained on benchmark datasets demonstrate that the paragraph vector-based method obtains superior per- formance over the existing approach in both identifying clusters and assigning ap- propriate descriptive labels to them. 1 Introduction Document clustering is a well-established tech- nique whose goal is to automatically organise a collection of documents into a number of seman- tically coherent groups. Descriptive document clustering goes a step further, in that each iden- tified document cluster is automatically assigned a human-readable label (either a word or phrase) that characterises the semantic content of the doc- uments within the cluster. Descriptive clustering methods have been shown to be useful in a va- riety of scenarios, including information retrieval (Bharambe and Kale, 2011), analysis of social net- works (Zhao and Zhang, 2011), and large-scale exploration (Nassif and Hruschka, 2013) and visu- alisation of text collections (Kandel et al., 2012). A number of previously proposed descriptive clustering techniques work by extending a stan- dard document clustering approach. Documents are typically clustered based on a bag-of-words (BoW) representation (i.e., the occurrence counts of the words that appear in each document). Then, each cluster is labelled using the most com- monly occurring word or phrase within the clus- ter (Weiss, 2006). In contrast to this approach, the recently proposed descriptive clustering ap- proach (CEDL) (Mu et al., 2016) maps documents and candidate cluster labels into a common se- mantic vector space (i.e., co-embedding). The co-embedding space facilitates the straightforward assignment of descriptive labels to document clus- ters. The CEDL method has been shown to gener- ate accurate cluster labels and achieved improved clustering performance when compared to stan- dard descriptive clustering methods. Nonetheless, the co-embedding is based solely on a BoW rep- resentation of the documents and is thus limited in its ability to accurately represent the semantic …

Content truncated to 500 words