Hyperbolic Relevance Matching for Neural Keyphrase Extraction

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5710 - 5720 July 10-15, 2022 ©2022 Association for Computational Linguistics Hyperbolic Relevance Matching for Neural Keyphrase Extraction Mingyang Song, Yi Feng and Liping Jing∗ Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University, China mingyang.song@bjtu.edu.cn Abstract Keyphrase extraction is a fundamental task in natural language processing that aims to ex- tract a set of phrases with important informa- tion from a source document. Identifying im- portant keyphrases is the central component of keyphrase extraction, and its main challenge is learning to represent information comprehen- sively and discriminate importance accurately. In this paper, to address the above issues, we design a new hyperbolic matching model (Hy- perMatch) to explore keyphrase extraction in hyperbolic space. Concretely, to represent in- formation comprehensively, HyperMatch first takes advantage of the hidden representations in the middle layers of RoBERTa and integrates them as the word embeddings via an adaptive mixing layer to capture the hierarchical syntac- tic and semantic structures. Then, considering the latent structure information hidden in natu- ral languages, HyperMatch embeds candidate phrases and documents in the same hyperbolic space via a hyperbolic phrase encoder and a hyperbolic document encoder. To discriminate importance accurately, HyperMatch estimates the importance of each candidate phrase by explicitly modeling the phrase-document rele- vance via the Poincaré distance and optimizes the whole model by minimizing the hyperbolic margin-based triplet loss. Extensive experi- ments are conducted on six benchmark datasets and demonstrate that HyperMatch outperforms the recent state-of-the-art baselines. 1 Introduction Keyphrase Extraction (KE) aims to extract a set of phrases related to the main points discussed in the source document, a fundamental task in Natu- ral Language Processing (NLP). Because of their succinct and accurate expression, keyphrase extrac- tion is helpful for a variety of applications such as information retrieval (Kim et al., 2013) and text summarization (Liu et al., 2009a). ∗Corresponding author. region large region region of a large region large region of region of land a large region of land . . . . . . 1-gram 2-gram 3-gram 5-gram … … The Great Plateau is a large region of land that is secluded from … Figure 1: Sample partial of the document in OpenKP dataset. For ease of presentation, we assume “a large region of land” is a 5-gram candidate phrase as an ex- ample in the document. Typically, most existing keyphrase extraction models mainly include two procedures: candidate keyphrase extraction and keyphrase importance es- timation. Specifically, the former extracts candi- date phrases from the document via some heuristics (Nguyen and Phan, 2009; Liu et al., 2009b; Grineva et al., 2009; Wan and Xiao, 2008; Liu et al., 2009c), and the latter determines which candidate phrases are keyphrases via unsupervised or supervised mod- els (Mihalcea and Tarau, 2004; Xiong et al., 2019; Sun et al., 2020; Song et al., 2021). The keyphrase importance estimation procedure usually plays a more critical role than …

Content truncated to 500 words