MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction

Findings of the Association for Computational Linguistics: ACL 2022, pages 396 - 409 May 22-27, 2022 c⃝2022 Association for Computational Linguistics MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction Linhan Zhang12∗ Qian Chen2 Wen Wang2 Chong Deng2 Shiliang Zhang2 Bing Li3 Wei Wang4 Xin Cao1 1School of Computer Science and Engineering, The University of New South Wales 2Speech Lab, Alibaba Group, China 3A*STAR Centre for Frontier AI Research (CFAR), Singapore 4Hong Kong University of Science and Technology (Guangzhou), China {linahan.zhang, xin.cao}@unsw.edu.au {tanqing.cq,w.wang,dengchong.d,sly.zsl}@alibaba-inc.com li_bing@ihpc.a-star.edu.sg weiwcs@ust.hk Abstract Keyphrase extraction (KPE) automatically ex- tracts phrases in a document that provide a concise summary of the core content, which benefits downstream information retrieval and NLP tasks. Previous state-of-the-art (SOTA) methods select candidate keyphrases based on the similarity between learned represen- tations of the candidates and the document. They suffer performance degradation on long documents due to discrepancy between se- quence lengths which causes mismatch be- tween representations of keyphrase candidates and the document. In this work, we pro- pose a novel unsupervised embedding-based KPE approach, Masked Document Embedding Rank (MDERank), to address this problem by leveraging a mask strategy and ranking can- didates by the similarity between embeddings of the source document and the masked doc- ument. We further develop a KPE-oriented BERT (KPEBERT) model by proposing a novel self-supervised contrastive learning method, which is more compatible to MDERank than vanilla BERT. Comprehensive evaluations on six KPE benchmarks demonstrate that the pro- posed MDERank outperforms state-of-the-art unsupervised KPE approach by average 1.80 F1@15 improvement. MDERank further bene- fits from KPEBERT and overall achieves aver- age 3.53 F1@15 improvement over the SOTA SIFRank. Our code is available at https: //github.com/LinhanZ/mderank. 1 Introduction Keyphrase extraction (KPE) automatically extracts a set of phrases in a document that provide a concise summary of the core content. KPE is highly beneficial for readers to quickly grasp the ∗Work is done during the internship at Speech Lab, Al- ibaba Group. key information of a document and for numerous downstream tasks such as information retrieval and summarization. Previous KPE works include supervised and unsupervised approaches. Super- vised approaches model KPE as sequence tagging (Sahrawat et al., 2019; Alzaidy et al., 2019; Mar- tinc et al., 2020; Santosh et al., 2020; Nikzad- Khasmakhi et al., 2021) or sequence generation tasks (Liu et al., 2020; Kulkarni et al., 2021) and require large-scale annotated data to perform well. Since KPE annotations are expensive and large- scale KPE annotated data is scarce, unsupervised KPE approaches, such as TextRank (Mihalcea and Tarau, 2004), YAKE (Campos et al., 2018), Em- bedRank (Bennani-Smires et al., 2018), are the mainstay in industry deployment. Among unsupervised KPE approaches, embedding-based approaches including Em- bedRank (Bennani-Smires et al., 2018) and SIFRank (Sun et al., 2020) yield the state-of-the- art (SOTA) performance. After selecting keyphrase (KP) candidates from a document using rule-based methods, embedding-based KPE approaches rank the candidates in a descending order based on a scoring function, which computes the similarity between embeddings of candidates and the source document. …

Content truncated to 500 words