An Empirical Study on Neural Keyphrase Generation

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4985–5007 June 6–11, 2021. ©2021 Association for Computational Linguistics 4985 An Empirical Study on Neural Keyphrase Generation Rui Meng† Xingdi Yuan‡ Tong Wang‡ Sanqiang Zhao† Adam Trischler‡ Daqing He† †School of Computing and Information, University of Pittsburgh ‡Microsoft Research, Montréal rui.meng@pitt.edu Abstract Recent years have seen a flourishing of neural keyphrase generation (KPG) works, including the release of several large-scale datasets and a host of new models to tackle them. Model per- formance on KPG tasks has increased signif- icantly with evolving deep learning research. However, there lacks a comprehensive com- parison among different model designs, and a thorough investigation on related factors that may affect a KPG system’s generalization per- formance. In this empirical study, we aim to fill this gap by providing extensive experi- mental results and analyzing the most crucial factors impacting the generalizability of KPG models. We hope this study can help clarify some of the uncertainties surrounding the KPG task and facilitate future research on this topic. 1 Introduction Keyphrases are phrases that summarize and high- light important information in a piece of text. Keyphrase generation (KPG) is the task of automat- ically predicting such keyphrases given the source text. The task can be (and has often been) easily misunderstood and trivialized as yet another natural language generation task like summarization and translation, failing to recognize one key aspect that distinguishes KPG: the multiplicity of generation targets; for each input sequence, a KPG system is expected to output multiple keyphrases, each a mini-sequence of multiple word tokens. Despite this unique nature, KPG has been essen- tially “brute-forced” into the sequence-to-sequence (Seq2Seq) (Sutskever et al., 2014) framework in the existing literature (Meng et al., 2017; Chen et al., 2018; Ye and Wang, 2018; Chen et al., 2019b; Yuan et al., 2020; Chan et al., 2019; Zhao and Zhang, 2019; Chen et al., 2019a).The community has ap- proached the unique challenges with much inge- nuity in problem formulation, model design, and evaluation. For example, multiple target phrases have been reformulated by either splitting into one phrase per data point or joining into a single se- quence with delimiters (Figure 1), both allowing straightforward applications of existing neural tech- niques such as Seq2Seq. In accordance with the tremendous success and demonstrated effective- ness of neural approaches, steady progress has been made in the past few years — at least empirically — across various domains, including sub-areas where it was previously shown to be rather difficult (e.g., in generating keyphrases that are not present in the source text). Meanwhile, with the myriad of KPG’s unique challenges comes an ever-growing collection of studies that, albeit novel and practical, may quickly proliferate and overwhelm. We are therefore mo- tivated to present this study as — to the best of our knowledge — the first systematic investigation on such challenges as well as the effect of inter- play among their solutions. We …

Content truncated to 500 words