Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1389–1404 August 1–6, 2021. ©2021 Association for Computational Linguistics 1389 Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention Wasi Uddin Ahmad†∗, Xiao Bai‡, Soomin Lee‡, Kai-Wei Chang† †University of California, Los Angeles, ‡Yahoo Research †{wasiahmad,kwchang}@cs.ucla.edu ‡{xbai,soominl}@verizonmedia.com Abstract Natural language processing techniques have demonstrated promising results in keyphrase generation. However, one of the major chal- lenges in neural keyphrase generation is pro- cessing long documents using deep neural net- works. Generally, documents are truncated be- fore given as inputs to neural networks. Conse- quently, the models may miss essential points conveyed in the target document. To overcome this limitation, we propose SEG-Net, a neural keyphrase generation model that is composed of two major components, (1) a selector that selects the salient sentences in a document and (2) an extractor-generator that jointly extracts and generates keyphrases from the selected sentences. SEG-Net uses Transformer, a self- attentive architecture, as the basic building block with a novel layer-wise coverage atten- tion to summarize most of the points discussed in the document. The experimental results on seven keyphrase generation benchmarks from scientific and web documents demonstrate that SEG-Net outperforms the state-of-the-art neu- ral generative methods by a large margin. 1 Introduction Keyphrases are short pieces of text that summa- rize the key points discussed in a document. They are useful for many natural language processing and information retrieval tasks (Wilson et al., 2005; Berend, 2011; Tang et al., 2017; Subramanian et al., 2018; Zhang et al., 2017b; Wan and Xiao, 2008; Jones and Staveley, 1999; Kim et al., 2013; Hulth and Megyesi, 2006; Hammouda et al., 2005; Wu and Bolivar, 2008; Dave and Varma, 2010). In the automatic keyphrase generation task, the input is a document, and the output is a set of keyphrases that can be categorized as present or absent keyphrases. Present keyphrases appear exactly in the target doc- ∗Work done during internship at Yahoo Research. Title: [1] natural language processing technologies for developing a language learning environment . Abstract: [1] so far , computer assisted language learning ( call ) comes in many different flavors . [1] our research work focuses on developing an integrated e learning environment that allows im- proving language skills in specific contexts . [1] integrated e learning environment means that it is a web based solution ..., for instance , web browsers or email clients . [0] it should be accessi- ble . . . [1] natural language processing ( nlp ) forms the technological basis for developing such a learn- ing framework . [0] the paper gives an overview ...[0] therefore , on the one hand , it explains cre- ation . . . [0] on the other hand , it describes existing nlp standards . [0] based on our requirements , the paper gives . . . [1] . . . necessary developments in e learning to keep in …

Content truncated to 500 words