Inductive Topic Variational Graph Auto-Encoder for Text Classification

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4218–4227 June 6–11, 2021. ©2021 Association for Computational Linguistics 4218 Inductive Topic Variational Graph Auto-Encoder for Text Classification Qianqian Xie Department of Computer Science, University of Manchester qianqian.xie@manchester.ac.uk Jimin Huang School of Computer Science, Wuhan University huangjimin@whu.edu.cn Pan Du Department of Computer Science and Operations Research, University of Montreal pandu@iro.umontreal.ca Min Peng School of Computer Science, Wuhan University pengm@whu.edu.cn Jian-Yun Nie Department of Computer Science and Operations Research, University of Montreal nie@iro.umontreal.ca Abstract Graph convolutional networks (GCNs) have been applied recently to text classification and produced an excellent performance. However, existing GCN-based methods do not assume an explicit latent semantic structure of doc- uments, making learned representations less effective and difficult to interpret. They are also transductive in nature, thus cannot han- dle out-of-graph documents. To address these issues, we propose a novel model named in- ductive Topic Variational Graph Auto-Encoder (T-VGAE), which incorporates a topic model into variational graph-auto-encoder (VGAE) to capture the hidden semantic information be- tween documents and words. T-VGAE inher- its the interpretability of the topic model and the efficient information propagation mecha- nism of VGAE. It learns probabilistic repre- sentations of words and documents by jointly encoding and reconstructing the global word- level graph and bipartite graphs of documents, where each document is considered individ- ually and decoupled from the global corre- lation graph so as to enable inductive learn- ing. Our experiments on several benchmark datasets show that our method outperforms the existing competitive models on supervised and semi-supervised text classification, as well as unsupervised text representation learning. In addition, it has higher interpretability and is able to deal with unseen documents. 1 Introduction Recently, graph convolutional networks (GCNs)(Kipf and Welling, 2017; Veliˇckovi´c et al., 2018) have been successfully applied to text classification tasks (Peng et al., 2018a; Yao et al., 2019; Liu et al., 2020; Wang et al., 2020). In addition to the local information captured by CNN or RNN, GCNs learn word and document representations by taking into account the global correlation information embedded in the corpus- level graph, where words and documents are nodes connected by indexing or citation relations. However, the hidden semantic structures, such as latent topics in documents (Blei et al., 2003; Yan et al., 2013; Peng et al., 2018b), is still ignored by most of these methods (Yao et al., 2019; Huang et al., 2019; Liu et al., 2020; Zhang et al., 2020), which can improve the text representation and pro- vide extra interpretability (in which the proba- bilistic generative process and topics make more sense to humans compared to neural networks, i.e. topics can be visually represented by top-10 or 20 most probable word clusters). Although few stud- ies such as (Wang et al., 2020) have proposed incor- porating a topic structure into GCNs, the topics are extracted in advance from the set of documents, in- dependently from the graph and information prop- agation among documents …

Content truncated to 500 words