Deep Attention Diffusion Graph Neural Networks for Text Classification

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8142–8152 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8142 Deep Attention Diffusion Graph Neural Networks for Text Classification Yonghao Liu†, Renchu Guan†, Fausto Giunchiglia‡, Yanchun Liang♦, Xiaoyue Feng†∗ †Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, College of Computer Science and Technology, Jilin University, China ♦Zhuhai Sub Laboratory, Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Zhuhai College of Science and Technology, China ‡DISI, University of Trento, Italy yonghao20@mails.jlu.edu.cn {guanrenchu, ycliang, fengxy}@jlu.edu.cn fausto.giunchiglia@unitn.it Abstract Text classification is a fundamental task with broad applications in natural language process- ing. Recently, graph neural networks (GNNs) have attracted much attention due to their pow- erful representation ability. However, most ex- isting methods for text classification based on GNNs consider only one-hop neighborhoods and low-frequency information within texts, which cannot fully utilize the rich context in- formation of documents. Moreover, these models suffer from over-smoothing issues if many graph layers are stacked. In this pa- per, a Deep Attention Diffusion Graph Neural Network (DADGNN) model is proposed to learn text representations, bridging the chasm of interaction difficulties between a word and its distant neighbors. Experimental results on various standard benchmark datasets demon- strate the superior performance of the present approach. 1 Introduction Text classification, as an important and fundamen- tal task in the field of natural language processing, has attracted extensive attention from scholars for many years and has been widely used in various practical applications, such as topic labeling (Wang and Manning, 2012), computational phenotyping (Che et al., 2015), question answering (Liu et al., 2015) and dialog act classification (Lee and Der- noncourt, 2016). The performance of text classi- fication relies heavily on the ability of the model to extract textual features from raw text. Previ- ous shallow learning-based text classification ap- proaches mainly use hand-crafted sparse lexical features, such as bag-of-words (BoW) or n-grams, for representing texts (Li et al., 2020). Since these features are predefined, the models do not take full advantage of the large amount of training data. Deep learning architectures represented by convo- lutional neural networks (CNNs) (Kim, 2014) and ∗Corresponding author. recurrent neural networks (RNNs) (Liu et al., 2015) are becoming more popular due to their strong per- formance in text mining. These models can capture semantic and syntactic information in local consec- utive word sequences well. Recently, graph neural networks (GNNs) have attracted increasing attention due to their superi- ority in dealing with complex structured data and relations (Kipf and Welling, 2017; Klicpera et al., 2019a). GNNs have achieved promising results in text classification tasks when modeling texts with graph structures due to their powerful expres- siveness (Wu et al., 2020). Despite the success of the mentioned models, several serious limita- tions of prevalent GNNs hinder their performance, which is mainly attributed to the following fac- tors: (I) Restricted Receptive Fields. Most previ- ous approaches allow a word in the graph to ac- …

Content truncated to 500 words