Autoencoding Keyword Correlation Graph for Document Clustering

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3974–3981 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3974 Autoencoding Keyword Correlation Graph for Document Clustering Billy Chiu Sunil Kumar Sahu Derek Thomas Neha Sengupta Mohammady Mahdy Inception Institute of Artificial Intelligence, Abu Dhabi, United Arab Emirates {hon.chiu|sunil.sahu|neha.sengupta}@inceptioniai.org, derek.thomas@pax-ai.ae, mohammady.mahdy@inceptioniai.org Abstract Document clustering requires a deep under- standing of the complex structure of long- text; in particular, the intra-sentential (local) and inter-sentential features (global). Exist- ing representation learning models do not fully capture these features. To address this, we present a novel graph-based representation for document clustering that builds a graph au- toencoder (GAE) on a Keyword Correlation Graph. The graph is constructed with top- ical keywords as nodes and multiple local and global features as edges. A GAE is em- ployed to aggregate the two sets of features by learning a latent representation which can jointly reconstruct them. Clustering is then performed on the learned representations, us- ing vector dimensions as features for induc- ing document classes. Extensive experiments on two datasets show that the features learned by our approach can achieve better clustering performance than other existing features, in- cluding term frequency-inverse document fre- quency and average embedding. 1 Introduction Text classification is a core task in natural language processing (NLP) with a variety of applications, such as news topic labeling and opinion mining. Supervised methods for text classification gener- ally perform better than unsupervised clustering methods, at the cost of heavy annotation efforts. In contrast, unsupervised clustering methods have the advantage in terms of requiring less prior knowl- edge and can be used to discover new classes when relevant training data is not available. The performance of text clustering is closely re- lated to the quality of its feature representation. While sentence-level clustering relies primarily on the local, intra-sentential features, document-level clustering also needs the global, inter-sentential features. Existing representation learning meth- ods that model text as a bag-of-words (e.g., term frequency-inverse document frequency, TFIDF) or as sequences of variable-length units (e.g., Bidirec- tional Encoder Representations from Transformers, BERT) (Devlin et al., 2019) are ineffective in cap- turing global features across long sequences – suf- fering from heavy computational cost as a result of high dimensionality and complex neural network architectures, as reported by Ye et al. (2017) and Jawahar et al. (2019). Recently, graph neural networks have been used to provide features for NLP applications, including text classification (Yao et al., 2019) and relation extraction (Sahu et al., 2019). By modeling text in a topological structure, these models can en- code global information in long-range words. De- spite their usefulness, graph models remain under- explored in document clustering. In this work, we propose a novel graph-based representation for document clustering by utiliz- ing a graph autoencoder (GAE) (Kipf and Welling, 2016) on a Keyword Correlation Graph (KCG). Our KCG represents a document as a weighted graph of topical keywords. Each graph node is a keyword, and sentences in the document …

Content truncated to 500 words