A Preliminary Exploration of GANs for Keyphrase Generation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8021–8030, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8021 A Preliminary Exploration of GANs for Keyphrase Generation Avinash Swaminathan1, Haimin Zhang2, Debanjan Mahata2, Rakesh Gosangi2, Rajiv Ratn Shah1, Amanda Stent2 1 IIIT-Delhi, India, 2 Bloomberg, New York s.avinash.it.17@nsit.net.in, {dmahata, hzhang449, rgosangi}@bloomberg.net, astent@bloomberg.net, rajivratn@iiitd.ac.in Abstract We introduce a new keyphrase genera- tion approach using Generative Adversar- ial Networks (GANs). For a given doc- ument, the generator produces a sequence of keyphrases, and the discriminator distin- guishes between human-curated and machine- generated keyphrases. We evaluated this ap- proach on standard benchmark datasets. We observed that our model achieves state-of-the- art performance in the generation of abstrac- tive keyphrases and is comparable to the best performing extractive techniques. Although we achieve promising results using GANs, they are not significantly better than the state- of-the-art generative models. To our knowl- edge, this is one of the first works that use GANs for keyphrase generation. We present a detailed analysis of our observations and ex- pect that these findings would help other re- searchers to further study the use of GANs for the task of keyphrase generation. 1 Introduction Keyphrases capture the most salient topics of a doc- ument and are often indexed in databases to help with search and information retrieval techniques. Researchers tag their scientific publications with high-quality keyphrases to ensure discoverability in scientific repositories. Automatic identification of keyphrases is of great interest to the scientific com- munity as it helps to recommend relevant articles, suggest missing citations to authors, identify po- tential peer reviewers, and analyze research trends (Augenstein et al., 2017). Keyphrases could either be extractive (part of the document) or abstractive (not part of the docu- ment). Some prior works have referred to them as present and absent keyphrases, respectively. Keyphrase generation is the process of predicting both extractive and abstractive keyphrases from a given document. Most of the previous works in keyphrase domain, including both supervised and unsupervised techniques, primarily focus on extractive keyphrases (Hasan and Ng, 2014; Ma- hata et al., 2018; Sahrawat et al., 2020). Recent studies Meng et al. (2017); Ye and Wang (2018); Chan et al. (2019) have started to develop genera- tive approaches that produce both abstractive and extractive keyphrases from documents. Though these studies have shown some promise, the results suggest that there is great room for improvement. Most of the supervised natural text generation approaches use Maximum Likelihood Estimation (MLE) objective (Lu et al., 2018). However, MLE techniques have often been observed not to be gen- erating satisfactory text because of exposure bias (Lu et al., 2018). Generative approaches based on Reinforcement Learning (RL) or adversarial training have been shown to address some of the challenges of exposure bias. One such example is the success in the field of summary generation using GANs (Wang and Lee, 2018). Driven by these developments, we posit that, as with sum- marization, keyphrase generation can also benefit from the use of GANs. …

Content truncated to 500 words