Heterogeneous Graph Neural Networks for Keyphrase Generation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2705–2715 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2705 Heterogeneous Graph Neural Networks for Keyphrase Generation Jiacheng Ye1∗, Ruijian Cai1∗, Tao Gui2†and Qi Zhang1† 1School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, China 2Institute of Modern Languages and Linguistics, Fudan University {yejc19, 19210240253, tgui, qz}@fudan.edu.cn Abstract The encoder-decoder framework achieves state- of-the-art results in keyphrase generation (KG) tasks by predicting both present keyphrases that appear in the source document and absent keyphrases that do not. However, relying solely on the source document can result in generating uncontrollable and inaccurate absent keyphrases. To address these prob- lems, we propose a novel graph-based method that can capture explicit knowledge from related references. Our model first retrieves some document-keyphrases pairs similar to the source document from a pre-defined index as references. Then a heterogeneous graph is constructed to capture relationships of different granularities between the source document and its references. To guide the decoding process, a hierarchical attention and copy mechanism is introduced, which directly copies appropriate words from both the source document and its references based on their relevance and significance. The experimental results on multiple KG benchmarks show that the proposed model achieves significant improvements against other baseline models, especially with regard to the absent keyphrase prediction. 1 Introduction Keyphrase generation (KG), a fundamental task in the field of natural language processing (NLP), refers to the generation of a set of keyphrases that expresses the crucial semantic meaning of a document. These keyphrases can be further categorized into present keyphrases that appear in the document and absent keyphrases that do not. Current KG approaches generally adopt an encoder- decoder framework (Sutskever et al., 2014) with attention mechanism (Bahdanau et al., 2015; Luong et al., 2015) and copy mechanism (Gu et al., 2016; ∗ Equal contribution. † Corresponding authors. NUS Krapivin SemEval KP20k 0 25 50 75 100 Percentage (%) Present Keyphrase Absent Keyphrase Figure 1: Proportion of present and absent keyphrases among four datasets. Although the previous methods for keyphrase generation have shown promising results on present keyphrase predictions, they are not yet satisfactory on the absent keyphrase predictions, which also occupy a large proportion. See et al., 2017) to simultaneously predict present and absent keyphrases (Meng et al., 2017; Chen et al., 2018; Chan et al., 2019; Chen et al., 2019b,a; Yuan et al., 2020). Although the proposed methods for keyphrase generation have shown promising results on present keyphrase predictions, they often generate uncon- trollable and inaccurate predictions on the absent ones. The main reason is that there are numerous candidates of absent keyphrases that have implicit relationships (e.g., technology hypernyms or task hypernyms) with the concepts in the document. For instance, for a document discussing “LSTM”, all the technology hypernyms like “Neural Network”, “RNN” and “Recurrent Neural Network” can be its absent keyphrases candidates. When dealing with scarce training data or limited model size, it is non- trivial for …

Content truncated to 500 words