Glocal: Incorporating Global Information in Local Convolution for Keyphrase Extraction

Proceedings of NAACL-HLT 2019, pages 1837–1846 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1837 Glocal: Incorporating Global Information in Local Convolution for Keyphrase Extraction Animesh Prasad†‡ Min-Yen Kan†‡ † School of Computing National University of Singapore ‡ Institute for Application of Learning Science and Educational Technology National University of Singapore animesh@u.nus.edu kanmy@comp.nus.edu.sg Abstract Graph Convolutional Networks (GCNs) are a class of spectral clustering techniques that leverage localized convolution filters to per- form supervised classification directly on graphical structures. While such methods model nodes’ local pairwise importance, they lack the capability to model global importance relative to other nodes of the graph. This causes such models to miss critical informa- tion in tasks where global ranking is a key component for the task, such as in keyphrase extraction. We address this shortcoming by allowing the proper incorporation of global information into the GCN family of models through the use of scaled node weights. In the context of keyphrase extraction, incorporating global random walk scores obtained from Tex- tRank boosts performance significantly. With our proposed method, we achieve state-of-the- art results, bettering a strong baseline by an ab- solute 2% increase in F1 score. 1 Introduction Learning directly on a graphical structure is a crucial requirement in many domains. These graphs represent information in many forms, rang- ing from interconnected user groups to contextu- ally linked documents to a central document by shared vocabulary. Learning on graphs has been studied extensively in the form of spectral cluster- ing (Ng et al., 2002). The potential of learning directly on graphs has realized in semi-supervised settings where labels for only a few of the nodes are available. Some prior work formulates such setup as propagating the label information using some form of graph-based regularization (Kipf and Welling, 2016). Recently proposed works have updated such methods to be end-to-end learn- able in the deep learning style by employing gra- dient descent on nodes within a fixed neighbor- hood, approximating spectral clustering’s means of approximating the graph’s eigenvectors (Bron- stein et al., 2017) by aggregating neighborhood features. Recent advancements in normalizing the gradient range further improve the efficiency of such solutions (Kipf and Welling, 2016). How- ever, these techniques can only exploit local fea- tures within the neighborhood of individual nodes. For some tasks, such simplified local feature ag- gregation may be sufficient, but insufficient for tasks that need global relative importance informa- tion. One such important graph-based task is keyphrase extraction. In this task, individual words or phrases serve as graph nodes, and edges represent some form of co-occurrence. Keyphrase extraction has been extensively studied, in both su- pervised (classification) and unsupervised (rank- ing) modes. Depending on the length of the text and the final application of the task, solutions can be sample-based classification, pairwise rank- ing or sequential labeling. For example, Kim et al.(2010) explore the case of extracting top keyphrases from complete documents for down- stream indexing, while Augenstein et al.(2017) connects its usage for knowledge base …

Content truncated to 500 words